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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  is  committed  to  strengthening  the  understanding  and  training  of  the  U.S. 
Army  command  and  control.  A  part  of  this  critical  task  is  to  develop  methods 
of  collecting  and  analyzing  command  and  control  performance  data  that  will 
help  identify  areas  that  could  benefit  from  enhanced  training  or  aiding 
techniques. 

This  paper  reports  on  the  modification  and  application  of  an  existing 
method  to  elicit  knowledge  about  Army  tactical  planning  exercises.  This  work 
is  particularly  valuable  because  of  the  depth  of  analysis  applied  to  the  plan¬ 
ning  session.  The  results  identify  potential  planning  process  problems  not 
typically  discovered  by  more  superficial  methods. 
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A  KNOWLEDGE  ELICITATION  STUDY  OF  MILITARY  PLANNING 


EXECUTIVE  SUMMARY 


Requirement: 

To  evaluate  whether  the  Critical  Decision  Method  jfCDM)  of  knowledge 
elicitation  can  be  used  in  Army  Command  and  Control  (C^)  exercises  as  a  means 
of  understanding  decision-making  dynamics.  Such  a  tool  could  provide  useful 
information  about  improving  procedures  and  training  as  well  as  for  specifying 
more  effective  decision  support  concepts. 


Procedure: 

The  CDM  was  used  during  three  different  planning  exercises.  Analysis  was 
focused  on  the  most  data  rich  and  realistic  of  these  exercises.  A  content 
analysis  was  performed  on  the  exercise  data,  generating  64  planning  segments 
during  a  5-hour  period.  Each  segment  was  coded  for  the  dominant  goals  and  the 
types  of  functions  being  performed.  These  data  present  a  chronological  record 
of  the  planning  session  organized  by  primary  themes. 


Findings: 

Fully  one-third  of  the  transitions  from  one  segment  to  the  next  were 
accidental,  created  by  the  typical  interruptions  of  a  simulated  battle  envi¬ 
ronment.  Once  the  interruption  was  completed,  planning  rarely  picked  up  from 
the  same  point.  Ninety-three  percent  of  the  information  used  was  obtained 
from  sources  immediately  in  the  planning  area  and  there  was  very  little  active 
searching  for  critical  pieces  of  information  that  were  not  directly  access¬ 
ible.  Twenty-six  out  of  twenty-seven  decisions  made  during  the  5-hour  session 
involved  no  comparison  between  options.  Many  of  the  probes  used  in  the  stan¬ 
dard  CDM  post-exercise  Interviews  were  redundant  with  the  planning  information 
data  gathered  during  the  sessions.  Moreover,  the  observations  and  tape  re¬ 
cordings  created  a  different  and  in  many  ways  more  accurate,  record  of  option 
generation  and  evaluation  than  the  retrospective  CDM  interviews.  On  the  other 
hand,  the  CDM  interviews  uniquely  obtained  data  about  levels  of  proficiency. 
Such  probes  have  been  very  informative  about  the  differences  between  profi¬ 
cient  and  novice  decision  makers,  and  the  data  directly  reflect  training  re¬ 
quirements.  Such  data  are  only  obtained  by  the  CDH,  not  by  direct  observa¬ 
tion.  Ve  also  developed  three  new  methods  for  data  analysis:  the  detailed 
content  analysis  of  a  planning  session  into  a  segmented  situation  assessment 
record;  the  overlaying  on  this  record  of  data  concerning  the  primary  activi¬ 
ties  of  each  segment;  and  a  technique  for  charting  progressive  deepening  bar¬ 
riers  that  are  identified,  and  strategies  for  overcoming  the  barriers. 


vii 


Utilization  of  Findings: 


The  findings  point  out  the  need  for  better  management  of  the  team  deci¬ 
sion  process  and  suggest  a  need  for  a  different  approach  to  decision-making 
training.  Army  centers  and  schools  responsible  for  command  and  control 
decision-making  training  as  veil  as  line  units  can  use  these  findings  to  help 
identify  training  requirements.  They  might  also  use  the  data  analysis  methods 
to  evaluate  performance  and  provide  training  feedback.  Further,  the  findings 
are  useful  for  command  and  control  system  designers  in  identifying  vhere  sys¬ 
tem  improvements  are  required. 
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A  KNOWLEDGE  ELICITATION  STUDY  OF  MILITARY  PLANNING 


Introduction 

This  report  presents  our  findings  about  the  use  of  the  Critical  Decision 
Method  (CDM)  for  describing  proficient  perforrance  within  a  battle  management 
setting.  We  undertook  this  project  to  learn  what  type  of  information  would  be 
obtained  using  CDM,  and  to  learn  how  we  would  need  to  expand  our  knowledge 
elicitation  methods  and  techniques  for  data  analysis. 

The  U.S.  Army  has  a  strong  interest  in  how  proficient  decision  makers  func¬ 
tion  in  complex  environments,  working  under  the  time  pressure  and  uncertain 
information  that  would  exist  on  a  modern  battlefield.  There  is  a  need  to  de¬ 
velop  a  knowledge  elicitation  method  that  can  be  used  in  operational  settings 
to  capture  decision-making  performance. 

The  CDM  is  a  knowledge  elicitation  technique  developed  under  the  sponsorship 
of  the  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  In 
1985,  Klein  Associates  was  awarded  a  Phase  I  Small  Business  Innovations  Re¬ 
search  (SBIR)  contract  to  study  time-pressured  decision  making.  The  research 
studied  urban  fire  ground  commanders,  analyzing  the  way  they  made  decisions 
about  the  allocation  of  personnel  and  resources.  The  results  showed  that  the 
great  majority  of  decisions  were  made  very  rapidly,  typically  in  less  than  one 
minute.  The  findings  (Klein,  Calderwood,  &  Clinton-Cirocco,  1986)  supported  a 
recognitional  model  of  decision-making. 

To  perform  the  study  we  developed  a  knowledge  elicitation  method  to  describe 
the  context-bound  expertise  of  commanders  with  over  20  years  of  experience. 

The  CDM  technique  we  developed  was  a  variant  of  Flanagan's  (1954)  critical 
incident  approach,  using  our  more  specialized  cognitive  probes  about  decision¬ 
making  strategies.  The  emphasis  of  CDM  is  on  non-routine  incidents  where  the 
differences  between  experts  and  novices  may  be  contrasted  most  clearly.  The 
probes  largely  deal  with  the  cues  relied  on  to  formulate  situation  assessment 
and  the  types  of  options  that  are  considered. 

Phase  II  of  the  SBIR  was  planned  to  accomplish  four  goals,  the  research 
presented  here  comprising  the  third.  The  first  goal  was  to  present  a  formal 
description  of  CDM  (Klein  &  MacGregor,  1988).  The  second  goal  uas  to  evaluate 
the  reliability  of  CDM.  The  reliability  has  been  shown  to  be  quite  high,  in 
the  range  of  60-902  agreement  between  independent  raters  (Taynor,  Crandall,  6 
Uiggin6,  1987).  The  third  goal  was  to  assess  whether  CDM  could  be  used  within 
the  constraints  of  an  Army  command  and  control  (C*)  exercise.  The  fourth  goal 
was  the  application  of  CDM  for  training  and  for  decision  support  development 
and  evaluation. 

The  following  sections  of  this  Introduction  will  describe  the  battle  manage¬ 
ment  environment  and  CDM,  and  briefly  review  some  earlier  research  studies 
using  CDM. 
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The  Battle  Management  Environment 


Battle  management  here  includes  the  tactical  planning  for  battle.  The  plan¬ 
ning  staff  consists  of  the  various  officers  and  their  staffs.  G3  (Operations) 
and  G2  (Intelligence)  personnel  are  the  main  participants  in  this  planning  * 

process,  but  G4  (Logi-stics),  G1  (Personnel),  and  all  other  staff  elements  take 
active  roles.  The  Commander  is  responsible  for  taking  all  staff  inputs  and 
coming  up  with  a  final  plan. 

Much  planning  occurs  well  ahead  of  any  battle,  and  the  larger  and  more 
likely  the  battle  the  greater  this  prior  planning.  However,  as  the  battle 
nears  there  is  always  significant  adjustment  to  the  original  plan  as  current 
conditions  evolve.  There  is  also  the  ongoing  requirement  to  conduct  the  battle 
Itself  and  to  perform  the  requisite  planning  during  the  engagement. 

We  have  given  these  phases  different  names,  as  they  appear  to  constitute 
quite  different  planning  and  decision-making  environments.  The  planning  that 
occurs  well  before  the  battle  we  identify  as  Long  Term  planning.  For  our  pur¬ 
poses  this  phase  can  be  considered  to  begin  from  months  to  years,  and  to  end 
from  days  to  weeks  before  the  actual  battle.  It  is  characterized  by  a  rela¬ 
tively  leisurely  pace  of  planning,  at  Inast  as  compared  to  the  following  two 
phases.  The  sort  of  contingency  planning  done  by  Pentagon  staffs  is  typical  of 
Long  Term  planning.  The  replanning  and  adjustments  to  current  conditions  that 
occur  just  prior  to  commencement  of  the  battle  we  identify  as  Near  Term  plan¬ 
ning.  This  takes  place  a  few  days,  up  to  perhaps  a  few  weeks,  prior  to  start 
of  a  battle.  It  is  characterized  by  increasing  time  pressures  and  an  increased 
flow  of  information  concerning  events  surrounding  the  immediate  battle  environ¬ 
ment.  Finally,  the  replanning  and  execution  of  the  battle  during  its  actual 
conduct  we  identify  as  Immediate  planning.  The  environments  that  were  investi¬ 
gated  and  reported  on  here  were  the  latter  two:  Near  Term  planning  and  Immedi¬ 
ate  planning. 

The  Near  Term  and  Immediate  battle  management  environments  are  characterized 
by  increased  confusion,  conflicting  information,  decreased  time  for  planning, 
and  an  overall  sense  of  immediacy  and  increased  pressure.  This  is  also  a 
highly  distributed  decision-making  environment  involving  numerous  individuals, 
each  with  a  functional  area  of  responsibility,  and  all  charged  with  working 
together  to  produce  a  coordinated  plan. 

The  Importance  of  Knowledge  Elicitation  in  the  Battle  Management  Environment 

Background  research  into  the  decision-making  literature,  (Klein,  in  press) 
indicated  the  difficulties  encountered  in  coming  to  grips  with  just  how  profi¬ 
cient  decision  makers  function  in  their  domain  Of  expertise.  Laboratory  ex¬ 
periments  with  domain  experts  have  been  disappointing  because  much  of  the 
richness  of  the  domain  cannot  be  carried  across  into  the  laboratory  setting. 

The  very  definition  of  experts  includes  the  ability  to  make  good  choices  in 
complex  and  confused  environments.  The  way  to  understand  and  analyze  experts 
doing  their  jobs  seems  to  consist  of  two  parts.  First,  the  analyst  oust  ob¬ 
serve  proficient  decision  makers  while  they  are  performing  their  tasks  in  the 
natural  environment.  Second,  the  analyst  nust  develop  and  employ  a  data  col¬ 
lection  that  is  as  invisible  and  unobtrusive  as  possible.  The  goal  is  to 


? 


elicit  as  nuch  detail  and  information  from  the  experts  as  possible,  while  leav¬ 
ing  the  tasks  unchanged.  It  is  usually  impossible  to  determine  from  observa¬ 
tion  alone  just  what  is  going  on  in  the  expert's  mind. 

The  distributed  nature  of  this  planning  and  decision-making  process  also 
deserves  comment.  Operational  planning  is  typically  a  group  process  and  the 
participants  may  be  at  several  different  physical  locations.  Radios  and  tele¬ 
phones  are  used  to  communicate  among  these  locations.  Periodically  all  staff 
elements  ere  brought  together  for  briefings  and,  as  necessary,  for  other 
events. 

We  will  now  present  a  brief  description  of  the  CDM  in  the  following  section. 
For  a  more  detailed  examination  of  the  method  see  MacGregor  and  Klein  (1988). 

Critical  Decision  Method 


The  Critical  Decision  Method  employed  in  this  study  was  developed  to  extract 
from  individuals  working  in  complex,  real  world  domains  the  important  turning 
points  in  domain-specific  situations,  i.e.,  the  ones  the  person  felt  were  crit¬ 
ical  to  the  eventual  resolution  of  the  situation. 

The  goals  of  CDM  are  to  be  both  as  non-intrusive  and  as  extractive  as  possi¬ 
ble  in  the  situation  and  to  draw  from  the  expert  or  novice  the  important  as¬ 
pects  of  his  knowledge  and  decision  making  in  the  domain.  CDM  was  originally 
developed  by  a  team  of  researchers  representing  the  disciplines  of  cognitive 
psychology,  communications,  and  anthropology. 

One  example  of  CDM  application  is  based  on  a  study  of  tank  platoon  leaders 
at  Ft,  Knox,  Kentucky,  in  1986-87.  The  results  of  this  study  have  been  re¬ 
ported  in  detail  in  Brezovic,  Klein,  &  Thordsen  (1987).  The  situation  was  the 
training  of  Army  tank  platoon  leaders,  junior  grade  officers  who  were  involved 
in  field  exercises  which  pitted  their  platoon  of  tanks  against  "enemy"  tanks. 
Accompanying  each  tank  crew  was  an  Army  non  commissioned  officer  (NCO)  who  was 
himself  a  tank  commander  and  had  considerable  experience  in  the  field.  These 
NCQs  functioned  as  instructors  and  would  be  considered  experts  in  this  domain. 

Researchers  accompanied  the  tank  crews,  with  one  researcher  riding  in  the 
platoon  leader's  tank,  observing  the  unfolding  action  and  commenting  into  his 
tape  recorder  when  appropriate.  These  comments  also  formed  the  basis  for  later 
knowledge  elicitation  and  critical  decision  identification  during  brief  inter¬ 
views  conducted  immediately  after  each  field  exercise.  Another  researcher 
accompanied  the  officer  instructor  who  set  up  and  executed  each  field  exercise. 

During  brief  breaks  after  each  exercise  one  researcher  would  interview  the 
novice,  and  the  second  researcher  would  interview  the  expert.  Both  would  em¬ 
ploy  a  set  of  CDM  probes  that  asked  a  number  of  questions  such  as!  What  do  you 
think  were  the  most  difficult  and  challenging  decisions  faced  in  this  exercise? 

If  you  had  to  rank  these  decisions  in  terms  of  learning  importance,  how  would 
you  rank  them?  For  each  decision  point,  what  did  you  do?  Suppose  you  had  to 
describe  the  situation  of  this  decision  to  a  friend  of  yours — would  you  say  to 
him  in  25  words  or  less?  Were  there  any  other  options  available  in  addition  to 
what  you  decided?  What  factors  would  have  made  you  reject  the  option  you  had 
chosen? 
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CDM  focused  on  the  critics!  points  in  the  situation  which  stand  out  in  the 
participants'  minds  as  particularly  important.  Then,  the  participants  were 
asked  to  describe  the  situation  surrounding  these  important  decision  points. 
Finally,  the  participants  were  asked  to  describe  what  cues  they  focused  on  to 
decide  upon  a  course  of  action.  Notice  that  we  have  moved  from  eliciting  the 
critical  decision  points,  to  eliciting  the  general  situation  surrounding  each 
critical  decision  point,  to  finally  eliciting  an  assessment  of  the  situation. 
This  entire  series  of  queries  is  the  CDM. 

CDM  as  currently  employed  has  evolved  to  the  point  where  it  can  now  be  used 
coincidentally  in  the  domain  being  investigated,  as  with  the  tank  study,  or 
after  the  fact,  based  on  interview  notes  and  audio  tape  recordings,  as  in  the 
current  study. 

One  important  finding  already  obtained  with  CDM  is  that,  as  hypothesized, 
the  experts  were  much  more  often  making  decisions  based  on  a  recognition-primed 
process,  snd  tach  less  often  by  generation  of  alternatives  and  simul taneous 
evaluation  (what  can  be  called  a  concurrent  deliberation  basis)*  Conversely, 
novices  in  identical  situations  are  much  more  likely  to  make  decisions  based  on 
the  generation  and  concurrent  evaluation  of  alternatives,  and  much  less  likely 
to  choose  solutions  on  a  serial,  recognition-primed  basis  (Klein,  in  press). 

In  the  battle,  management  domain,  it  seems  important  to  recognize  that  we 
have  experts  making  time-critical  decisions*  in  a  distributed  decision  frame¬ 
work.  The  importance  of  applying  CDM  in  this  complex,  time-critical  domain  is 
to  help  determine  training  agendas  that  will  more  rapidly  and  efficiently  up¬ 
grade  the  skills  of  novices  to  levels  of  competence  and  proficiency. 

Method 

The  CDM  was  applied  in  three  settings:  at  Ft.  Leavenworth,  Kansas  in  a 
division  and  corps  classroom  setting,  at  Ft.  Riley,  Kansas  in  a  field  training 
brigade  exercise,  and  at  Ft.  Hood,  Texas  in  a  computer  driven,  battalion  exer¬ 
cise. 


Ft.  Leavenworth 

I  iim,  111111— W— — *  K.MWWWIWI.V 

This  was  a  classroom  training  exercise  at  the  Command  and  General  Staff 
College  located  at  Ft.  Leavenworth,  Kansas.  The  observations  occurred  over  a 
three  day  period  in  December  1936.  The  setting  was  corps  and  division  planning 
based  upon  an  American  geographical  scenario.  There  were  approximately  1." 
individuals  in  the  class.  Most  participants  were  majors  in  the  tegular  U.S, 
Army.  Exceptions  included  one  Air  Force  captain,  one  Marine  major  and  one 
Greek  Regular  Army  colonel* 

The  training  scenario  involved  the  planning  of  a  counterattack  against,  So¬ 
viet  Threat  forces  that  were  occupying  the  United  States.  The  students  had 
approximately  two  days  to  develop  a  plan  for  the  counterattack. 

One  primary  goal  for  this  data  collection  was  t.o  give  the  researchers  direct 
experience  with  some  of  the  issues  and  terminology  involved  in  the  military 
planning  process.  As  with  most  knowledge  elicitation  methods,  a  certain  amount 
of  domain  information  is  required  and  this  was  a  first  attempt  to  gain  some. 
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Two  researchers  observed  the  exercise.  One  focused  upon  the  events  that 
took  place  within  the  G3  section  (operations  and  plans)  and  the  other  focused 
upon  the  G2  section  (intelligence).  Notes  were  taken  and  briefings  were  au¬ 
dio-taped.  The  researchers  conducted  sort  interviews  using  CDM  when  the  plan¬ 
ners  within  these  sections  took  breaks.  Examples  of  the  two  interview  guides 
employed  in  these  sessions  for  the  G2  and  G3  are  in  Appendix  A. 

In  our  opinion,  the  CDM  worked  well  capturing  the  processes  taking  place, 
with  one  necessary  qualification.  Our  desire  was  to  apply  CDM  in  a  realistic 
planning  environment.  This  operation  was  a  classroom  exercise  to  train  plan¬ 
ning  skills.  Therefore,  we  can  only  comfortably  say  that  a  classroom  process 
was  captured.  Although  the  setting  addressed  corps  and  division-level  planning, 
it  contained  several  artificial  elements  and  lacked  some  realistic  ones  which 
affected  the  functional  nature  of  the  process.  Some  of  these  were  the  absence 
of  real  and  continuous  time  planning,  and  the  presence  of  "game  rule"  changes 
for  the  sake  of  instruction,  all  of  which  affected  the  overall  sense  of  realism 
of  the  exercise. 

Nevertheless,  the  observation  was  profitable  in  helping  confirm  that  the  CDM 
approaches  used  for  collecting  data  in  previous  studies  could  be  modified  for 
the  battle  management  planning  environment. 

Ft.  Riley 

This  was  a  National  Guard  field  exercise  conducted  at  Ft.  Riley,  Kansas. 

The  observations  occurred  over  a  three-day  period  in  May  and  June  of  1987.  The 
setting  was  a  brigade  field  tactical  operations  center  (TOC)  and  the  exercise 
was  based  upon  a  Western  European  geographical  scenario.  The  exercise  was 
driven  by  a  combat  board  training  game.  A  combined  S3  and  S2  section  was  ob¬ 
served.  Most  participants  were  captains  and  majors  in  the  National  Guard. 

The  context  of  the  training  scenario  was  the  planning  of  an  offensive  maneu¬ 
ver  against  Soviet  Threat  forces.  The  battle  took  place  over  a  two-day  period. 

Two  researchers  observed  the  exercise.  The  initial  plan  was  to  have  one 
focus  upon  the  events  taking  place  within  the  S3  (operations  and  plans)  and  the 
other  upon  the  S2  (intelligence).  Notes  were  taken  and  occasional  briefings 
were  audio-taped.  When  the  planners  had  available  time,  the  researchers  con¬ 
ducted  short  CDM  interviews.  Examples  of  the  interview  guides  used  for  these 
sessions  are  in  Appendix  A. 

Although  this  exercise  avoided  some  of  the  limitations  of  the  Ft.  Leaven¬ 
worth  classroom  setting,  it  raised  another  difficulty.  Simultaneous  with  this 
exercise,  another  maneuver  was  being  conducted  and  many  of  the  personnel  as¬ 
signed  to  this  operation  were  abruptly  reassigned  to  the  other,  leaving  this 
brigade  TOC  understaffed.  There  was  one  officer  who  was  present  throughout  the 
entire  exercise.  Often  he  was  the  only  individual  working  in  either  intelli¬ 
gence  or  operations.  He  assumed  responsibility  for  S3  and  S2  tasks  as  well  as 
for  occasional  air  and  fire  support.  The  result  was  that  most  of  his  time  was 
spent  monitoring  the  progress  of  the  "battle"  with  virtually  no  time  available 
for  planning. 
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We  found  this  trip  profitable  since  we  were  able  to  use  various  aspects  of 
the  exercise  to  modify  CDM  to  better  suit  the  military  planning  environment. 

Ft.  Hood 


This  was  a  simulation-driven  training  exercise  of  a  Regular  Army  infantry 
battalion  conducted  at  Ft.  Hood,  Texas.  The  observations  occurred  over  a  three 
day  period  in  June  of  1987.  The  setting  was  the  battalion  TOC  and  the  exercise 
was  based  upon  a  Western  European  geographical  scenario.  There  were  upwards  of 
25  participants  in  the  TOC  as  well  as  another  25  individuals  running  the  exer¬ 
cise  from  the  computer  center.  At  any  given  time  the  TOC  staffing  averaged 
four  to  five  individuals  in  the  S3  shop  and  two  to  three  in  the  S2  shop.  Most 
participants  were  captains  and  majors. 

The  context  of  this  training  scenario  was  the  execution  of  an  attack  against 
Soviet  Threat  forces  on  day  one,  the  development  of  a  defensive  plan  on  day 
two,  and  the  execution  of  that  plan  on  day  three. 

This  exercise  was  devoid  of  most  of  the  problems  encountered  in  the  first 
two  trips.  The  scenario  was  driven  by  the  Army  Training  Battle  Simulation  Sys¬ 
tem  (ARTBASS),  a  computer-based,  training  aid  that  allows  the  input  and  maneu¬ 
vering  of  resources  (friendly  and  enemy)  over  a  digitized  terrain  map.  The 
system  can  track  all  phases  of  the  battle  and  determine  the  outcome  of  fire 
fights,  artillerv  and  air  missions,  ammunition  used  and  remaining,  speed  of 
movement,  etc.  It  should  be  noted  that  the  individuals  in  the  battalion  TOC, 
who  were  set  up  in  a  simulated  field  setting,  did  not  have  access  to  these  ma¬ 
chines.  The  computers  were  in  another  location  and  information  was  input  and 
read  by  individuals  serving  in  the  roles  of  the  brigade  commander  (above  the 
battalion)  and  the  company  commanders  (below  the  battalion).  The  messages  be¬ 
tween  the  brigade  and  company  commanders  (both  of  whom  were  interfacing  with 
the  computer)  and  the  battalion  TOC  (no  computer)  were  transmitted  via  radio. 

Two  researchers  observed  the  exercise.  One  researcher  focused  upon  the  ac¬ 
tivities  of  the  S3  section  and  the  other  watched  the  S2  section.  Notes  were 
taken  and  occasional  briefings  were  audio-taped.  When  the  planners  had  availa¬ 
ble  time,  the  researchers  conducted  interviews  using  the  CDM.  Examples  of 
interview  guides  used  in  these  sessions  are  in  Appendix  A. 

Data  were  collected  for  all  three  stages  of  the  planning  mentioned  above. 

One  particular  stage  (five  hours  of  developing  a  defensive  battle  plan)  was 
extremely  rich  and  conducive  to  the  knowledge  elicitation  method  and  subsequent 
analysis. 

During  the  course  of  this  stage,  most  of  the  planning  was  conducted  by  open 
discussion  among  two  to  ten  individuals  who  were  gathered  around  the  S3  map. 

The  researchers  placed  a  small  tape  recorder  directly  in  front  of  the  map  and 
recorded  the  entire  course  of  discussion  from  which  the  plan  emerged.  Addi¬ 
tionally,  during  short  break  periods,  available  participants  were  questioned 
about  aspects  of  the  plan.  It  should  be  noted  that  during  this  planning  ses¬ 
sion,  while  S3  was  amply  staffed,  the  S2  section  often  consisted  of  only  one  or 
two  individuals.  The  result  was  that  although  we  were  able  to  record  the  ac¬ 
tual  planning  of  the  S3  section,  the  intelligence  work  often  involved  one 
individual  examining  the  S2  map  and  making  silent  observations.  Therefore  the 
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majority  of  the  data  are  from  the  S3  section  perspective.  Although  additional 
data  were  collected,  it  was  decided,  due  to  differences  in  the  nature  of  these 
data,  to  center  the  primary  analysis  around  the  transcripts  from  the  five  hours 
of  S3  planning. 

Data  Analysis 

At  this  point  we  faced  a  decision  about  allocating  our  own  resources.  We 
had  collected  a  large  amount  of  data  during  the  trips  to  Ft.  Leavenworth  and 
Ft.  Riley.  Most  of  these  data  had  not  been  analyzed  by  the  time  we  had  the 
opportunity  to  observe  the  ARTBASS  exercise  at  Ft.  Hood.  The  data  collected  at 
Ft.  Hood  turned  out  to  be  the  richest  source  of  data  of  all. 

One  alternative  was  to  try  to  analyze  some  data  from  each  of  the  observation 
sites.  Another  was  to  abandon  the  data  from  the  first  two  sites,  and  concen¬ 
trate  on  the  Ft.  Hood  data*  We  chose  the  latter  option. 

Our  justification  was  that  the  Ft.  Hood  exercise  was  the  most  realistic,  the 
closest  to  typical  Arny  exercises,  and  the  one  from  which  we  could  learn  the 
most.  The  Ft.  Leavenworth  and  Ft.  Riley  exercises  were  each  limited  in  impor¬ 
tant  ways.  Moreover,  CDM  data  had  already  been  successfully  collected  at  Ft. 
Leavenworth  and  Ft.  Riley.  We  had  been  able  to  conduct  the  interviews  as  in 
our  own  earlier  research,  to  ask  the  same  questions,  and  use  the  same  probes. 

If  anything,  these  data  collection  efforts  were  even  easier  than  the  Ft.  Knox 
study  where  we  had  been  limited  to  10-minute  interviews  with  the  tank  platoon 
leaders.  We  knew  that  we  could  do  the  same  data  analyses  that  we  had  used 
earlier,  cataloguing  critical  cues  and  aspects  of  situational  assessment  that 
emerged  through  the  interviews. 

For  these  reasons  we  chose  to  concentrate  all  of  the  analytical  resources  on 
the  Ft.  Hood  exercise.  We  wanted  to  study  the  audio  tapes  of  the  planning 
session  and  to  develop  new  methods  of  analysis  to  handle  these  types  of  data. 

Our  goal  shifted  from  simply  demonstrating  that  we  could  conduct  CDM  inter¬ 
views,  to  pushing  our  analytical  techniques  further  to  capture  real-time  plan¬ 
ning  dynamics. 

A  transcription  was  made  of  the  five-hour  planning  session.  These  trans¬ 
cripts  served  as  a  direct  timeline  of  events  from  the  defense  planning  session, 

A  general  flow  of  the  planning  session  began  to  emerge  during  the  analysis  of 
the  transcripts.  Each  phase  of  the  analysis  was  developed  independently  via 
input  from  researchers  who  were  at  the  exercise  and  those  who  were  not.  The 
Intent  of  using  two  Independent  analyses  was  to  provide  an  unbiased  check  on 
the  quality  of  the  products. 

The  first  level  of  analysis  involved  providing  a  simple  timeline  of  iden¬ 
tifiers  which  could  be  used  to  follow  the  flow,  and  also  locate  information  in 
the  transcripts.  Sequencing  and  continuity  were  deemed  the  most  important 
properties  to  be  preserved  by  this  process.  Long  sheets  of  computer  paper  were 
used  to  "map  out"  the  development  of  the  plan.  This  enabled  us  to  visualize 
sequencing  as  well  as  similar  references  of  information  through  time.  Further, 
it  served  as  the  basis  for  what  later  will  be  referred  to  as  the  "segments"  of 
the  planners'  objectives. 
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There  are  important  factors  in  military  settings  that  affect  how  the  plans 
are  developed:  missing  information,  uncertainty  with  existing  information,  and 
confusion  as  to  the  state  of  the  existing  plan.  Capturing  these  elements  may 
be  one  of  the  key  issues  which  is  not  addressed  by  traditional  approaches  to 
military  and  corporate  decision  making.  By  developing  a  visual  representation 
of  the  process  as  it  evolved  we  were  one  step  closer  to  identifying  these  fac¬ 
tors.  Therefore,  our  next  step  in  the  analysis  was  to  monitor  the  focus  of 
planners  through  time  and  to  attempt  to  label  these  foci  in  terms  that  were 
specific  to  the  military  domain. 


We  proceeded  to  track  the  general  flow  of  events  through  time,  and  then 
attempted  to  break  out  the  content-specific  foci  the  planners  were  addressing. 
Through  the  initial  readings  of  the  transcripts,  a  few  of  the  elements  became 
obvious.  For  example: 


•  friendly  strategy:  What  about  this,  I've  got  an  obstacle  here,  it's 

an  open  area  for  Delta  [company]  to  come  in  and 
engage... 

•  engineering  efforts:  I  [the  engineer]  can  give  you  three  battle 

positions,  but  I'm  going  to  need  at  least  two  more 
dozers. • . 


•  scouting  efforts:  [Scouts]  will  be  on  a  forward  screen  line,  early 

warning.. . 

•  commander's  intent:  The  Colonel  [Commander]  wants  to  defend  in  depth, 

so  I'm  just  giving  these  guys  areas  to  orient  on 
to  fire... 


Elements  which  were  not  so  obvious  also  began  to  emerge  when  this  categoriz¬ 
ing  of  foci  was  employed.  Examples  of  these  include: 

•  problem  recognition:  We're  going  to  corner  him  [a  friendly  element], 

we're  talking  lives  here. 

•  general  planning  Issues:  You  want  me  to  throw  this  movement  overlay 

here? 


The  real  difficulty  lay  in  not  forcing  elements  into  categories  yet  exhaus¬ 
tively  representing  the  "real  concerns"  and  foci  of  the  planners.  Operation¬ 
alizing  these  elements  was  an  iterative  process  which  required  many  hours  of 
deliberation. 


Next  we  tried  to  track  direct  references  to  information  in  the  transcripts 
and  then  see  if  they  reoccurred  and  if  they  did  whether  they  reoccurred  within 
the  same  context.  To  accomplish  this  it  was  necessary  to  number  each  unit  of 
information  as  it  occurred  in  the  transcripts,  then  evaluate  whether  the  infor¬ 
mation  was  already  In  existence  from  a  previous  context  or  discussion. 

It  is  from  these  analytical  approaches  that  the  following  results  were  ob¬ 
tained.  An  excerpt  from  the  data  map  of  the  five  hour  planning  session  is 
presented  in  Appendix  C. 
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Results 


The  analysis  focused  on  the  following  key  areas: 

2 

(a)  Results  of  the  C  Analysis. 

1.  Segments  of  the  Plan  and  Segment  Objectives. 

2.  Reference  Units  and  Reference  Unit  Foci. 

3.  Functional  Processes  Employed  by  the  Planners. 

A.  Higher  Order  and  Sub-Goals. 

5.  Decision  Making. 

6.  Option  Generation. 

7.  Information  Seeking  and  Acquisition. 

8.  Nature  of  Transitions  Between  Segments. 

9.  Progressive  Deepening. 

(b)  Assessment  of  CDM. 

1.  Critical  Decision  Methodology  Applications  in  Military  Setting. 

2.  Modeling  Group  Decision  Making. 

2 

Results  of  the  C  Analysis 

Segments  of  the  plan  and  segment  objectives.  There  were  12  primary  areas  of 
focus  (objectives)  on  which  the  planners  concentrated.  Each  occupied  from  1-to 
20-minute  periods  averaging  approximately  4-1/2  minutes.  These  can  be  viewed 
as  the  themes  of  sections  of  the  planning  process.  For  example  planners  may 
have  spent  10  minutes  working  out  a  subplan  for  the  use  of  obstacles.  This 
effort  did  not  generate  the  entire  obstacle  plan;  it  was  the  objective  on  which 
they  focused  their  attention  during  that  period  of  time.  This  10  minute  period 
would  be  classified  as  a  segment  of  functional  focus  with  "obstacles"  being  its 
objective.  If  thlB  same  objective  was  readdressed  later,  it  would  be  treated 
as  a  separate  segment,  even  though  the  nature  of  the  objective  would  be  the 
same  (obstacles).  In  other  words,  if  there  are  two  occasions  where  they  deal 
with  obstacles  it  was  coded  as  two  separate  segments  sharing  an  objective. 
Appendix  B  provides  a  sample  of  the  transcript  with  segments  marked. 

The  operational  definitions  for  coding  the  segment  objectives  were  as 
follow: 

COMMANDER'S  INTENT:  A  direct  attempt  to  interpret  and/or  satisfy  the 

commander's  intent. 

OBSTACLES:  Planning  the  use  and  placement  of  obstacles. 

ROUTES  AND  POSITIONS:  Determining  appropriate  routes  in  and  out  of  the 

battle  areas  and  potential  positions  for  the  friendly  elements. 

RECONNAISSANCE,  SCOUTS  AND  FORWARD  PATROLS;  The  use,  withdrawal,  or  safety 

of  reconnaissance,  scouts,  and  forward  patrols. 
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RESOURCES:  Determining  the  availability  of  adequate  quantities  of  resources 
including  tanks,  trucks,  mines,  personnel,  air  support,  artillery, 
bulldozers,  etc. 

ENEMY  HYP0THET1CALS  AND  PLANS:  Consideration  of  potential  enemy  strategy, 
movements,  resources,  attributes,  and  options. 

ENGINEERING:  Consideration  of  the  engineering  work  necessary  to  establish 
the  friendly,  pre-battle  layout  of  obstacles,  positions,  demolitions,  etc. 

FIRE  SUPPORT:  The  establishment  or  use  of  fire  support  such  as  artillery. 

AIR  SUPPORT:  The  establishment  or  use  of  air  resources  and  support. 

FRIENDLY  STRATEGY  AND  HYPOTHETICALS:  Offensive  or  defensive  planning  and 
contingencies  concerning  the  friendly  forces  under  the  command  of  this 
headquarters. 

MOVEMENT  OF  FRIENDLY  TROOPS:  The  movement  of  friendly  troops  from  a 
previous  battle  sector  into  the  current  sector’s  primary  battle  positions. 

GENERAL  PLANNING  AND  "HOUSEKEEPING":  The  process  mechanics  of  developing 
the  plan. 

Table  1  shews  the  breakdown  of  the  number  of  segments  which  concentrated  on 
each  of  the  12  objectives. 


Table  1 

Frequencies  of  Segment  Objectives 


Objective 

Number  of  Segments 

Friendly  strategy  and  hypothetical 

14 

Obstacles 

9 

General  planning  and  "housekeeping" 

9 

Engineering 

8 

Recon,  scouts,  and  forward  patrols 

6 

Commander's  intent 

5 

Routes  and  p>  .it ions 

5 

Fire  support 

4 

Enemy  hypothetical  and  plans 

3 

Resources 

2 

Movement  of  friendly  troops 

2 

Air  support 

1 

10 


To  help  examine  the  sequence  of  the  segment  objectives  through  the  course  of 
plan  development,  the  objectives  were  mapped  onto  the  main  data  sheet  (Appendix 
C).  As  might  be  expected,  the  planners  did  not  always  address  these  objectives 
separately.  There  were  64  individual  segments.  In  60  of  the  segments,  single 
objectives  were  "in  the  spotlight"  (e.g.,  obstacles,  resources,  etc.)  while  in 
the  remaining  four,  double  objectives  were  addressed  (e.g.,  obstacles  and  engi¬ 
neering,  friendly  strategy  and  enemy  strategy,  etc.).  There  did  not  appear  to 
be  any  set  pattern  as  to  which  objective  would  follow  any  other. 

In  addition  to  the  segments  and  objectives  described  above,  the  transcript 
data  were  categorized  within  eight  additional  parameters.  First  the  tran¬ 
scripts  were  divided  into  230  discrete  units  which  were  labeled  reference 
units.  These  were  the  smallest  units  of  the  transcripts  which  retained  any 
context  of  the  discussions.  Each  of  these  units  generally  had  at  least  one  of 
18  specific  foci.  These  reference  units  and  their  foci  will  be  addressed  in  the 
next  section. 

Each  of  the  reference  units  were  also  examined  and  coded  for  the  types  of 
operational  processes  taking  place  in  them.  It  was  found  that  the  activities 
could  be  categorized  into  any  of  six  types  of  processes:  introduction  of  in¬ 
formation,  appraisal  of  information,  generation  of  options,  decision  making, 
refinement  of  previous  decisions,  and  general  housekeeping  or  clarification 
activities.  These  process  categories  are  defined  in  a  later  section  along  with 
the  analysis  performed  upon  them. 

Other  analyses  were  conducted  examining  the  goal  structures  employed  by  the 
planners,  the  nature  of  (reasons  for)  transitions  between  the  segment  objec¬ 
tives,  and  the  issue  of  progressive  deepening.  Following  these  transcript 
analyses,  CDM  was  assessed  along  two  dimensions:  its  applications  in  military 
settings  and  its  use  in  modeling  distributed  decision  making.  It  should  be 
clarified  that  while  the  military  planning  environment  is  considered  distrib¬ 
uted,  these  data  came  from  one  observed  central  location  in  the  process  (the 
S3  section).  These  analyses  and  assessments  will  now  be  addressed  separately. 

Reference  units  and  reference  unit  foci.  While  the  64  segments  of  the  plan 
were  the  primary  focus  of  our  analysis,  we  divided  the  entire  transcript  into  a 
number  of  smaller  units  to  simplify  referencing  the  transcripts  and  to  aid  in  e 
more  detailed  analysis.  These  "reference  unitB"  generally  embodied  verbal 
interactions  regarding  a  specific  subject,  or  focus.  They  differed  from  seg¬ 
ments  in  that  a  segment  consisted  of  all  contiguous  discussions  intended  to 
meet  some  objective,  such  as  planning  the  movement  of  a  friendly  unit.  Ref¬ 
erence  units,  on  the  other  hand,  consisted  of  more  narrow  topics  and  topic 
changes  within  a  segment.  For  example,  discussing  the  scouting  and  reconnais¬ 
sance  of  possible  routes  might  be  a  reference  unit  within  the  broader  segment 
of  planning  the  movement  of  a  friendly  unit. 

These  reference  units  consisted  of  sections  ranging  from  one  sentence  to 
four  or  five  paragraphs.  The  average  was  about  one  paragraph  of  transcript. 

The  final  total  of  reference  units  was  230  with  the  average  unit  accounting  for 
approximately  1.25  minutes  of  planning  (range:  13  seconds  to  10  minutes). 


11 


Although  these  breaks  were  somewhat  arbitrary,  we  attempted  to  have  them  repre¬ 
sent  the  smallest  units  of  the  planning  that  could  be  identified  from  the  tran¬ 
scripts.  The  64  segments  generally  included  from  2  to  15  of  these  reference 
units.  An  example  of  a  section  of  the  transcript  with  its  segments  and  ref¬ 
erence  units  labeled  is  included  in  Appendix  B. 

While  the  planners  concentrated  on  the  12  objectives  of  the  previously  men¬ 
tioned  segments,  their  moment-by-moment  attention  switched  between  18  smaller 
areas  of  focus  within  the  reference  units.  Each  reference  unit  generally  had  at 
least  one  of  these  issues  on  which  it  focused.  You  will  find  some  of  the  ob¬ 
jectives  and  foci  with  duplicate  labels  because  there  was  overlap  in  the  con¬ 
tent  of  the  12  objectives  and  the  18  areas  of  focus.  For  example,  while  in  the 
middle  of  a  ten  minute  discussion  about  obstacle  placement  (the  objective)  the 
planners  briefly  focused  on  the  meaning  of  the  commander's  intent — "He  wants  us 
to  mine  this  avenue"  (the  area  of  focus).  Here  the  focus  (commander's  intent) 
was  within  the  context  of  the  objective  (obstacle  placement).  At  a  later  time 
the  objective  of  a  different  ten  minute  discussion  was  completely  about  the 
commander's  intent,  e.g.,  the  commander  was  present  and  gave  additional  direc¬ 
tion.  In  the  former  segment,  the  commander's  intent  was  one  of  the  18  areas  of 
focus,  while  in  the  latter  it  was  the  segment  objective. 

These  reference  units  and  reference  unit  foci  can  be  thought  of  as  minor, 
subareas  of  discussion  within  the  segments  that  in  and  of  themselves  are  not 
complete  enough  to  stand  on  their  own.  For  example,  a  segment  may  involve  a 
ten  minute  discussion  about  moving  a  battalion  from  location  A  to  location  B, 

The  reference  unit  and  reference  unit  foci  may  include  a  one  minute  subdiscus¬ 
sion  concerning  the  use  of  check  points,  another  two  minute  sub  discussion  con¬ 
cerning  specific  routes  to  take  and  so  forth.  The  reference  units  are  not  com¬ 
plete  enough  to  "stand  on  their  own"  and  they  only  have  meaning  and  impact 
within  the  framework  of  the  overall  section. 

Working  directly  from  the  transcript,  coders  identified  the  foci  of  each 
reference  unit.  The  operational  definitions  for  the  coding  of  these  foci  are 
as  follow; 

COMMANDER'S  INTENT:  The  giving  of  orders  by  the  commander  or  the  6taffs 
attempt  to  interpret  those  orders. 

FRIENDLY  STRATEGY;  References  to  offensive  or  defensive  planning  and  con¬ 
tingencies  of  the  friendly  forces  under  the  command  of  this  headquarters. 

TIMING;  Specific  references  to  issues  where  the  time  of  execution  is  a 
critical  element  to  success. 

SCOUTS  AND  RECONNAISSANCE;  References  to  issues  concerning  scouting,  recon¬ 
naissance,  forward  observation  and  the  safety  of  these  efforts. 

OBSTACLES;  Reference  to  any  object  or  barrier,  whether  natural  or  man-made, 
that  the  planners  were  considering  using  as  an  Impediment  to  the  movement  of 
enemy  forces. 

TERRAIN:  Natural  characteristics  of  the  land  including  topography,  rivers, 
vegetation,  etc. 
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RESOURCES:  Specific  references  to  availability  of  adequate  quantities  of 
resources  including  tanks,  trucks,  mines,  personnel,  air  support,  artillery, 
bulldozers,  etc. 

ENGINEERING  EFFORTS:  References  to  the  engineering  work  necessary  to  estab¬ 
lish  the  friendly,  pre-battle  layout  of  obstacles,  positions,  demolitions, 
etc. 

ROUTES:  Considerations  of  avenues  of  ingress  to  and  egress  from  the  battle 
area. 

POSITIONING:  References  to  placement  of  specific  elements  on  the  primary 
and  subsequent  battle  areas. 

ENEMY  STRATEGY:  References  regarding  the  potential  strategy  and  options 
that  enemy  forces  may  employ  and  how  this  might  be  affected  by  the  elements' 
size,  number,  and  attributes;  the  avenues  of  approach;  resources;  terrain; 
etc. 

OTHER  FRIENDLY  FORCES:  Consideration  of  other  friendly  elements  in 
adjoining  sectors. 

GENERAL  PLANNING  AND  "HOUSEKEEPING":  Issues  concerning  the  process  mechan¬ 
ics  and  semantic  elements  of  developing  the  plan. 

SOVIET  DOCTRINE:  References  to  standard  anticipated  actions  of  the  enemy 
forces  as  prescribed  by  friendly  military  manuals  and  training. 

LOGISTICS:  Issues  which  relate  to  stocking,  equipping,  and  transporting 
supplies,  resources,  materials,  etc.  to  the  friendly  elements. 

ARTBASS:  Direct  reference  to  the  rules  and  limitations  imposed  on  the  exer¬ 
cise  scenario  by  the  computerized  Army  Training  and  Battle  Simulation  Sys¬ 
tem. 

WEATHER:  Direct  reference  to  influences  of  weather. 

RECOGNITION  OF  PROBLEMS:  Identification  of  potential  problems. 

Table  2  shows  the  relative  percentages  of  occurrence  of  these  foci,  in  order 
of  appearance  during  the  planning  process. 
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Table  2 


Frequency  of  Occurrence  of  Reference  Unit  Foci 


Foci 

Frequency 

Percent 

Friendly  strategy 

107 

20.0 

Terrain 

48 

9.0 

Obstacles 

45 

8.4 

Commander's  intent 

43 

8.0 

Routes 

37 

6.9 

Resources 

36 

6.7 

General  planning  and  "housekeeping" 

36 

6.7 

Enemy  strategy 

31 

5.8 

Positioning 

30 

5.6 

Scouts  and  recon 

28 

5.2 

Engineering 

28 

5.2 

Timing 

20 

3.7 

Logistics 

14 

2.6 

Recognition  of  problems 

14 

2.6 

Other  friendly  forces 

5 

0.9 

ARTBASS  (scenario  driver  rules) 

5 

0.9 

Weather 

5 

0.9 

Soviet  doctrine 

4 

0.7 

Total 

536 

100.0 

Not  surprisingly,  the  area  to  which  the  planner's  attention  was  directed 
most  often  was  friendly  strategy  (20%),  followed  by  terrain  (9%),  obstacles 
(8,4%)  and  commanders  intent  (8%).  General  planning  and  "housekeeping"  issues 
appeared  more  often  than  enemy  strategy,  almost  twice  as  often  as  timing  is¬ 
sues,  and  nearly  three  times  more  than  logistic  and  recognition  of  problem 
issues.  Weather,  other  friendly  forces  and  Soviet  doctrine  were  nearly  non¬ 
existent.  It  is  interesting  to  note  that  recognition  of  problems  was  the  last 
foci  introduced.  For  example:  if  the  planners  follow  through  with  a  particu¬ 
lar  option  for  the  friendly  movement  from  one  location  to  another,  many  of  the 
troops  will  get  lost  if  they  are  not  provided  checkpoints*  This  appeared  ap¬ 
proximately  2  hours  into  the  planning  process. 

Functional  processes  employed  by  the  planners.  We  identified  six  categories 
of  the  processes  taking  place  within  each  reference  unit  of  the  transcripts. 
These  included  a)  the  introduction  of  information  (new  and  old),  b)  the  Genera¬ 
tion  of  options,  c)  the  appraisal  of  information,  options,  etc.,  d)  decisions 
made,  e)  the  refinement  of  previously  made  decisions,  and  f)  clarification  of 
issues,  processes,  etc.  which  did  not  advance  the  plan  in  any  way. 

We  coded  each  reference  unit  by  the  functional  processes  which  took  place, 
the  order  in  which  they  took  place,  and  repetitions.  The  operational  defini¬ 
tions  used  for  the  coding  of  these  categories  with  examples  follow: 


Information:  Any  pieces  of  information  introduced  into  the  planning 
environment*  This  information  does  not  have  to  be  used  by  the  planners,  as 
long  as  it  is  technically  available  for  their  use  and  if  used,  could  further 
the  state  of  the  plan.  For  example:  "(We  have)  four  planes,  two  on  strip 
alert  which  is  30  minutes,  two  on  standby  alert  which  is  immediate." 

Generation:  The  generation  of  options  and  alternatives  by  the  planners. 

For  example:  "You  drop  (artillery)  here,  here,  here,  here... up  until  the 
time  when  you  don't  want  to  drop  it  anymore." 

Appraisal:  Any  general  discussions,  debates,  arguments,  etc.  that  the 
planners  took  part  in,  that  served  to  further  the  state  of  the  plan.  This 
sometimes  Included  "kicking  a  dead  horse"  discussions,  since  we  judge  these 
helped  the  individual  planners  further  their  understanding  of  the  plan  even 
if  it  was  something  of  a  lost  cause.  For  example:  "(It  depends)  on  where 
you  want  to  put  it,  whal.  the  defenses  look  like,  direction  of  wind,  hazard 
conclusion  analysis  for  chemical-also  because  once  we  throw  it  they  are 
going  to  throw  it." 

Decisions:  Any  decision  that  is  clearly  made  with  the  authority  of  rank  or 
with  the  consensus  of  the  decision  group  supporting  it.  For  example:  "No, 
send  them  down  to,  up  out  of  bounds,  right  there,  back  up,  stop,  cut  them 
off  right  there,  straight  down  into  the  section. .." 

Refinement:  Any  "fine  tuning"  of  a  previously  made  decision.  For  example: 
from  above  decision, 

A:  You  want  the  road  junction  right  here? 

B:  Yeah,  800. 

A;  800,  870. 

B:  How  do  you  all  give  out  routes,  do  you  want  to  give  out  a  route  like 
that  overlay? 

A:  We'll  probably  have  to,  that's  a  pretty  substantial  move. 

Clarification:  Any  process  which  primarily  serves  a  purpose  of  general 
"housekeeping"  or  covering  "old  ground"  for  someone  who  Just  came  into  the 
session,  etc.  A  requirement  of  this  category  is  that  it  does  nothing  to 
further  the  current  state  of  the  plan.  For  example: 

A:  Eb  you  want  a  map  size  overlay  of  the  route  or  do  you  want  m  to  draw  it 
on  an  8  x  11  sheet  of  paper? 

B:  No,  put  it  on  something  that  they  can  put  on  a  piece  of  acetate  that 
they  can  put  on  their  map. 

Table  3  gives  the  frequencies  of  occurrence  of  these  six  categories. 
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Table  3 


frequencies  of  Functional  Planning  Processes 


Category 

Frequency 

Percent 

Appraisal 

225 

38.8 

Information 

164 

28.2 

Generation 

102 

17.6 

Clarification 

40 

6.9 

Decision 

27 

4.7 

Refinement 

22 

3.8 

Total 

580 

100.0 

Two-thirds  of  the  planning  process  involved  introduction  of  information  and 
appraisal,  while  almost  one-fifth  Involved  generation  of  options.  The  remain¬ 
ing  15%  was  divided  among  decision  making,  refinement  of  the  decisions,  and 
clarifications. 

Table  3  reflects  only  the  frequencies  of  occurrence,  not  the  amount  of  plan¬ 
ning  time  occupied  by  each  of  these  processes.  The  judgment  of  the  researchers 
for  the  relative  magnitude  of  planning  time  required  for  the  six  categories, 
is; 


APPRAISE  >  GENERATE  >  INFORMATION  *  REFINE  -  CLARIFY  >  DECISION 

Higher  order  and  sub-goals.  The  planners  worked  within  a  framework  of  two 
overriding  goals: 

(1)  Deny  the  Enemy  Phase  Line  Hawaii. 

(2)  Deny  the  Enemy  Phase  Line  Hawaii  by  killing  them. 

While  these  were  undeniably  the  overriding  goal*,  they  were  of  such  a  high 
order  that  they  were  not  often  directly  addressed.  We  therefore  chose  to  look 
at  the  sub-goals  operating  under  these  two  primary  goals. 

Within  the  two  overriding  goals,  the  planners  were  driven  by  eight  sub- 
goals,  most  of  which  had  cultiple  sub-sub-goals*  The  eight  sub-goals  and  their 
sub-sub-goals  were  arrived  at  by  identifying  the  specific  goal  of  each  refer¬ 
ence  unit  (the  smallest  unit  of  the  transcript).  This  was  accomplished  throut, « 
four  readings  of  the  transcript.  The  first  pass  was  "open"  in  that  the  coders 
began  with  no  predetermined  goal  categories  and  identified  the  goals  that  each 
reference  unit  seemed  to  satisfy.  The  second  pass  reassigned  units  to  more 
appropriate  specific  goal  categories.  This  was  necessary  in  light  of  the  fact 
that  the  first  pass  generated  additional  categories  as  the  coding  progressed. 
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The  third  and  fourth  passes  were  conducted  to  assure  the  coder  that  the  appro¬ 
priate  goal(s)  was  assigned  to  each  reference  unit.  The  eight  identified  sub¬ 
goals  were: 

(1)  Determine  where  the  enemy  was  coming. 

(2)  Determine  when  the  eneny  was  coming. 

(3)  Slow  the  enemy’s  advance. 

(4)  Canalize  the  enemy. 

(5)  Allow  the  withdrawal  of  friendly  troops. 

(6)  Move  friendly  troops  to  new  battle  positions. 

(7)  Establish  the  friendly  defensive  strategy. 

(8)  Accommodate  the  Commander. 

Table  4  outlines  the  entire  goal  structure  with  accompanying  statements  of 
the  outlined  goals.  The  frequencies  and  percentages  are  presented  in  Table  5. 
Note  that  the  values  presented  in  "Each"  column  of  Table  5  do  not  overlap.  For 
example,  there  were  12  occurrences  where  the  sub-goal  was  "determining  when  the 
enemy  was  coming."  These  were  separate  occurrences  from  the  10  when  the  sub¬ 
goal  was  to  "use  scouts  to  determine  when  the  enemy  was  coming,"  and  so  forth. 
That  is,  there  were  times  when  a  sub-goal  was  discussed  without  mentioning 
specific  sub-goals.  The  "Sum"  columns  however  reflect  the  summation  of  all 
occurrences  within  a  particular  sub-goal. 


Table  4 


Goals  and  Goal  Statements 


Outline 


Goal  Statement 


(Deny  P/L  Hawaii)  Do  not  let  enemy  reach  Phase  Line  Hawaii 

(Kill  enemy)  Kill  enemy  to  keep  the.-  from  P/L  Hawaii 

(With  Indirect)  Use  indirect  fire  -  kill  eneny  to  deny  P/L  Hawaii 

(With  Direct)  Use  direct  fire  -  kill  enemy  to  deny  P/L  Hawaii 


1*  Where  Enemy  coming 

1.1  Use  Scouts 

1.2  S2 

1.3  Terrain 

2.  When  Enemy  coming 

2.1  Use  Scouts 

2.2  Use  S2 


1.  Determine  where  enesy  is  coming 

1.1  Use  Scouts  to  determine  where  enemy  is 
coming 

1.2  Use  S2  to  determine  where  ene&y  is 
coming 

1*3  Analyze  terrain  to  determine  where 
enemy  is  coming 

2.  Determine  when  enemy  is  coming 

2.1  Use  scouts  to  determine  when  enemy  is 
coming 

2.2  Use  S2  to  determine  when  eneny  is 
coming 


3*  Slow  Enemy 

3.1  Use  obstacles 

3.1.1  Use  Bridges 

3.1.2  Mines/FASCAM 

3.2  Use  artillery 
3,2.1  Chemical 


A.  Canalize  Enemy 
A.l  Use  obstacles 
A, 1.1  Use  Bridges 

A. 1.2  Mines/FASCAM 

A. 2  Terrain 
A. 3  Artillery 


3.  Slow  the  advance  of  enemy 

3.1  Use  obstacles  to  slow  advance  of  enemy 

3.1.1  Turn  bridges  into  obstacles  to  slow 
enemy 

3.1.2  Use  mlnes/FASCAM  as  obstacles  to 
slow  enemy 

3.2  Use  artillery  to  slow  enemy 

3.2.1  Use  chemical  artillery  to  slow  the 
advance  of  enemy 

A.  Canalize  enemy 

A.l  Use  obstacles  to  canalize  enemy 

A. 1,1  Turn  bridges  into  obstacles  to 
canalize  enemy 

A.  1.2  Use  mines/FASCAM  as  obstacles  to 
canalize  enemy 

A. 2  Use  natural  terrain  to  canalize  enemy 

4.3  Use  artillery  to  canalize  enemy 


5.  Allow  withdraw 

5.1  Use  obstacles 

5.2  Artillery 

5.3  Routes 

5. A  Scouts  Withdrawal 


5.  Allow  friendlies  to  withdrawal 

5.1  Use  obstacles  to  help  friendlies 
withdraw 

5.2  Use  artillery  to  help  friendlies 
withdraw 

5.3  Determine  routes  for  friendly  withdraw 
5. A  Allow  friendly  scouts  to  withdraw 


IS 


Table  4  (Continued) 


Outline 


Goal  Statement 


6.  Forward  Movement 

6.1  Routes 

6.1.1  Checkpoints 

6.2  Resources 


6.  Allow  forward  movement  of  friendly  units 

6.1  Determine  routes  for  forward  movement 
of  friendly  units 

6*1*1  Use  checkpoints  to  mark  routes  for 
forward  movement 

6.2  Determine  resources  necessary  for 
forward  movement 


7.  Establish  Friendly  Defense  7. 

7.1  Use  Terrain 

7.2  Use  Obstacles 

7.3  Use  Engineer 

7.3.1  Protect  Engineer 


Establish  friendly  defensive  position 

7.1  Use  terrain  to  help  establish  friendly 
defense 

7*2  Use  obstacles  to  help  establish 
friendly  defense 

7.3  Use  engineering  assets  to  help 
establish  friendly  defense 
7.3.1  Protect  engineers  establishing 
friendly  defense  positions 


8.  Accommodate  Cmdr 


8. _ Please  the  commander 


Table  5 


Frequencies  of  Sub-Goals  and  Sub-Sub-Goal s 


Goal 

Frecuency 

Sum  Each 

Percentage 

Sum  Each 

1.  Where  Enemy  coming 

26 

12 

10.42 

4.82 

1.1  Use  Scouts 

10 

4.0 

1.2  S2 

2 

0.8 

1*3  Terrain 

2 

0.8 

2.  When  Enemy  coining 

10 

2 

4.02 

0.82 

2.1  Use  Scouts 

7 

2.8 

2.2  Use  S2 

1 

0.4 

3.  Slow  Enemy 

35 

5 

14.02 

2.02 

3.1  Use  obstacles 

7 

2.8 

3.11  Use  Bridges 

7 

2.8 

3.12  Mines/FASCAM 

11 

4.4 

3.2  Use  artillery 

3 

1.2 

3.21  Chemical 

2 

0.8 

4.  Canalize  Enemy 

45 

9 

18.02 

3.62 

4.1  Use  obstacles 

8 

3.2 

4.11  Use  Bridges 

15 

6.0 

4.12  Mines/FASCAM 

7 

2.8 

4.2  Terrain 

4 

1.6 

4,3  Artillery 

2 

0.8 

5.  Allow  withdraw 

29 

8 

11.62 

3.22 

5.1  Use  obstacles 

2 

0.8 

5.2  Artillery 

1 

0.4 

5.3  Routes 

16 

6.4 

5.4  Scouts  Withdrawal 

2 

0.8 

6.  Forward  Movement 

26 

12 

10.42 

4,82 

6.1  Routes 

6 

2.4 

6.1,1  Checkpoints 

4 

1.6 

6.2  Resources 

4 

1.6 

7.  Estimate  Friendly  Defense 

75 

42 

30.02 

16.82 

7.1  Use  Terrain 

8 

3.2 

7.2  Use  Obstacles 

10 

4.0 

7.3  Use  Engineer 

14 

5.6 

7.3,1  Protect  Engineers 

1 

0.4 

S.  Accommodate  Cmdr 

-i 

4 

1.62 

1.62 

Total 

250 

250 

100.02 

100.02 
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Decision  making.  There  were  27  decision  points  identified  from  the  tran¬ 
scripts  of  the  five-hour  planning  session.  The  decisions  were  coded  with  re¬ 
spect  to  whether  or  not  they  represented  Recognition-Primed  Decisions  (RPDs). 
Klein  (in  press)  has  distinguished  between  recognitional  and  analytical  deci¬ 
sions.  Recognitional  decisions,  also  termed  RPDs,  occur  when  a  decision  maker 
recognizes  a  situation  as  typical,  recognizes  the  typical  reaction  to  that 
situation,  evaluates  .the  reaction  for  feasibility,  and  then  either  implements 
it,  improves  it,  or  rejects  it  for  another  reaction.  "Typical"  in  this  context 
is  used  to  refer  to  a  situation  that  the  decision  maker  has  previously  experi¬ 
enced  or  has  been  taught  to  expect  or  anticipate.  In  contrast,  analytical 
decisions  involve  the  generation  of  a  set  of  options  that  are  compared  to  each 
other  on  a  set  of  evaluation  dimensions.  This  strategy  is  based  on  concurrent 
deliberation  of  options.  We  coded  our  data  by  determining  whether  each  deci¬ 
sion  point  involved  some  form  of  concurrent  deliberation  of  multiple  options 
(non-RPD)  or  if  it  involved  only  serial  examination  of  one  option  at  a  time 
(RPD). 

Twenty-six  of  the  decisions  were  classified  as  RPDs  while  only  one  was  ana¬ 
lytical.  That  is,  of  the  27  decision  points  observed  during  the  five-hour 
exercise,  only  once  did  the  planners  contrast  two  or  more  options.  The  classi¬ 
cal  laboratory  decision  model  has  generally  examined  concurrent  decisions  based 
upon  novel  domains.  It  is  fairly  evident  that  in  this  particular  planning 
exercise,  there  was  little  applicability  of  classical  models  (for  example,  see 
Slovic,  Fischoff,  &  Lichtenstein,  1977)  since,  for  these  personnel  working 
within  a  familiar  domain,  serial  or  recognitional  processes  accounted  for  96% 
of  the  decisions. 

Table  6  describes  the  27  decisions  as  well  as  indicating  the  reference  unit 
location  and  approximate  chronological  placement  in  the  planning  process. 
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Table  6 


Decision  Point  Descriptions 


Decision  Reference  Approximate 

Point  Unit  Chronological 

Number _ Location _ Location _ Decision  Point  Description 


1 

2 

3  min 

2 

3 

4  min 

3 

7 

9  min 

4 

24 

30  min 

5 

28 

35  min 

6 

40 

50  min 

7 

47 

59  min 

8 

48 

60  min 

9 

54 

68  min 

10 

56 

70  min 

11 

61 

76  min 

12 

61 

76  min 

13 

91 

121  min 

14 

99 

124  min 

15 

104 

130  min 

16 

108 

135  min 

17 

108 

135  min 

18 

108 

135  min 

19 

108 

135  min 

20 

112 

140  min 

21 

157 

196  min 

22 

184 

230  min 

*23 

192 

240  min 

24 

195 

244  min 

25 

211 

264  min 

26 

220 

275  min 

27 

220 

275  min 

Place  guard  to  see  bridge  blown 
Put  observation  forward 
Assign  priority  of  efforts 
Everyone  clear  out  of  room 
Block  this  road 
Select  unit  battle  position 
Engineers  to  proceed  with  plan 
Dig  tank  positions  here 
Leave  scout  to  guard  bridge 
Get  12  hours  setup  prep  time 
Hove  the  units  on  this  route 
Place  the  RP  here 
Do  not  use  checkpoints 
Hake  this  the  engagement  area 
Request  two  dozers 
Blow  these  three  bridges 
Assign  battle  positions  and  EAs 
Put  battle  positions  along  RR 
Send  dozers  north  and  work  down 
Put  decon  sites  in  this  area 
Crater  road  and  FASCAM  clearing 
Have  BP  fire  into  this  EA 
Recon  to  identify  enetqy  route 
Seal  woods  with  scout  and  abatis 
Place  B,  C,  &  D  Companies  here 
Dig  four  battle  positions  here 
Put  wire  here  to  make  them  turn 


♦This  is  the  only  decision  where  multiple  options  were  considered* 


Option  generation*  The  transcripts  were  coded  to  identify  the  generation  of 
options  throughout  the  planning  session.  A  total  of  102  options  were  identi¬ 
fied.  The  majority  of  these  were  simply  mentioned  during  the  planning  session 
and  were  not  analyzed  during  any  of  the  27  decision  points  discussed  above. 
These  options  were  divided  into  three  categories: 


o  Those  that  the  researchers  could,  with  confidence,  say  were  incorporated 
into  the  final  working  plan  (n"45), 

o  Those  that  either  were  not  used  or  those  that  the  researchers  were  not 
certain  of  having  being  Included  in  the  final  plan  (nsS0). 

o  Those  that  were  Commander's  options  and  not  actually  generated  by  the 
planners  (n“7). 
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Since  it  is  generally  assumed  that  a  decision  is  necessary  to  "select"  an 
option,  an  attempt  was  made  to  match  the  27  decisions  to  the  45  options  that  we 
felt  confident  did  appear  in  the  final  plan.  The  result  was  that  20  (44%)  of 
the  options  could  be  traced  to  specific  acts  that  could  be  considered  as  deci¬ 
sions  while  25  (56%)  appear  to  have  become  part  of  the  plan  without  any  overt 
decision  making.  Possibly  the  officer  in  charge  of  operations  made  these  deci¬ 
sions  without  requiring  group  discussion  so  they  did  not  show  on  our  trans¬ 
cripts.  Another  possibility  is  that  some  of  these  were  "accidental  decisions," 
i.e.,  these  options  became  part  of  the  plan  without  anyone  consciously  making 
the  decision. 

Information  seeking  and  acquisition.  There  were  164  discrete  pieces  of  in¬ 
formation  brought  to  the  attention  of  the  planners  that  we  could  identify  from 
the  five  hours  of  transcripts.  The  manner  in  which  this  information  was  ob¬ 
tained  was  classified  as  either  "active"  or  "passive".  The  active  category 
Included  any  Information  that  was  requested  while  the  passive  category  referred 
to  information  that  was  not  requested  such  as  material  volunteered  by  other  in¬ 
dividuals  or  that  was  available  within  the  planning  environment.  Sixty-nine 
percent  (n“113)  of  the  information  acquisition  was  passive  and  thirty-one  per¬ 
cent  (n«*51)  was  active. 

The  164  pieces  of  information  were  also  coded  as  to  the  nature  of  their 
source.  Seven  sources  were  identified  as  follows:  individuals,  the  current 
state  of  the  plan,  the  map,  the  operations  order,  the  commander,  brigade,  and 
Soviet  doctrine.  The  operational  definitions  used  by  the  coders  for  these 
sources  with  examples  are  as  follows: 

Individuals:  People  who  were  physically  in  the  planning  room  who  possessed 
the  information  and  passed  it  on  either  voluntarily  or  upon  a  request.  This 
could  also  include  information  passed  from  subordinate  units  to  planners 
prior  to  their  coming  into  the  planning  area  under  observation.  For 
example,  from  an  individual's  personal  experience  in  the  area,  he  said: 

"These  roads  are  not  going  to  be  marshy." 

The  Plan:  Information  available  to  the  planners  by  virtue  of  the  current 
state  of  the  plan.  This  generally  is  the  information  the  planners  added  to 
the  map  overlays.  This  is  not  to  be  confused  with  the  basic  information 
available  from  the  map  itself.  For  example,  a  player  said  while  examining 
the  plan  overlay  of  the  map:  "Not  good.  (There  is]  nothing  to  keep  the 
enemy  In  there. • .no  obstacles." 

The  Hap:  Information  which  is  inherently  available  to  the  planners  from  the 
map  of  the  area  of  concern.  This  information  generally  involved  terrain 
features,  routes,  natural  obstacles  and  barriers,  etc.  This  should  not  be 
confused  with  aspects  of  planning  generated  by  the  planners  that  would  be 
found  on  the  map  overlay.  For  example:  "There  are  four  bridges  in  this 
area." 

The  Operations  Order  (0P0RDER):  Information  that  is  available  to  the 
planners  from  the  orders  they  have  received  about  the  operation  they  are 
planning.  This  would  involve  material  such  as  the  current  location  of 
friendly  units,  sector  boundaries,  or  other  friendly  units  in  neighboring 
sectors.  For  example:  "We  have  two  days  to  set  up." 
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The  Commander:  Information  that  is  spelled  out  by  the  Commander  in  his 
intent  statement.  For  example:  "Delta  Company  goes  here." 

Brigade:  Information  that  is  acquired  from  brigade  headquarters  (the  next 
HQ  level  above  the  individuals  in  this  exercise).  For  example;  after 
contacting  brigade  a  player  said:  "There  was  no  nuclear  release." 

Soviet  Doctrine:  Information  that  is  cited  from  the  materials  and  handbooks 
about  Soviet  Doctrine.  For  example:  "The  [enemy]  will  come  down  the  major 
roads... this  one  or  this  one." 

Table  7  shows  the  frequencies  and  percentages  of  these  sources  of  acquisi¬ 
tion  of  information  by  active,  passive  and  total  categories. 


Table  7 

Sources  of  Acquisition  of  Information 


Source  of  Active  Passive  Total 

Information _ Number  Percent _ Number  Percent _ Number  Percent 


Individuals 

19 

37.3% 

50 

44.2% 

69 

42.1% 

Plan 

17 

33.3 

8 

7.1 

25 

15.2 

Map 

6 

11.8 

52 

46.0 

58 

35.4 

Op  Orders 

5 

9.8 

1 

0.9 

6 

3.7 

Commander 

2 

3.9 

1 

0.9 

3 

1.8 

Brigade 

2 

3.9 

0 

0.0 

2 

1.2 

Soviet  Doctrine 

JO 

0.0 

1 

0.9 

1 

0.6 

Totals 

51 

100.0% 

113 

100.0% 

164 

100.0% 

More  than  two-thirds  of  the  information  was  passively  acquired,  with  over 
92 %  of  it  being  from  extremely  accessible  sources  such  a9  individuals  in  the 
room  or  the  map.  The  actively  acquired  information  constituted  one-third  of 
the  total  Information,  with  70%  of  it  also  being  from  very  accessible  sources 
(individuals  or  the  current  plan).  The  battalion  intelligence  (S2)  section  as 
a  source  of  Information  is  glaringly  missing.  There  was  no  one  from  the  S2 
section  present  in  the  S3  section  during  this  time  and  the  S3  section  did  not 
solicit  any  information  from  the  S2  section.  In  general,  if  the  information 
was  not  immediately  available  it  was  not  acquired. 

Nature  of  transitions  between  segments.  The  nature  of  the  transitions  be¬ 
tween  the  objective  segments  was  examined.  Eleven  types  of  breaks  were  identi¬ 
fied.  They  are  as  follow: 
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In-Context  Appraisals  (1CA):  Transitions  which  occur  because  discussions 
which  are  in  the  context  of  the  current  focus  redirect  the  planners  onto  a 
new  focus.  For  example,  the  planners  were  discussing  friendly  strategy  when 
the  observation,  "I've  got  to  cover  the  whole  sector  cause  I  don't  know 
where  (the  enemy]  is  going  to  come.  These  fallback  positions  [aren't  very 
good]..."  redirected  the  objective  to  the  engineering  efforts  necessary  to 
improve  the  fallback  positions. 

In-Context  Questions  (ICQ):  Transitions  which  occur  because  a  question 
which  is  in  the  context  of  the  current  focus  redirects  the  planners  onto  a 
new  focus.  For  example,  the  planners  were  discussing  friendly  strategy 
about  the  use  of  artillery  and  mine  placements  when  the  question  "What  do  we 
have  for  engineering  assets,  how  much  mines  have  we  used  and  what  have  we 
got  now?"  redirected  the  objective  to  engineering  assets  and  efforts. 

Out-Of-Context  Appraisal  (OCA):  Transitions  which  occur  because  discussions 
which  are  out  of  the  context  of  the  current  focus  redirect  the  planners  onto 
a  new  focus.  For  example,  the  planners  were  discussing  engineering  efforts 
when  the  observation  "I've  got  an  order  to  put  out  at  1300"  redirected  the 
objective  to  plan  "housekeeping"  issues. 

Out-Of-Context  Questions  (OCQ):  Transitions  which  occur  because  a  question 
which  is  out  of  the  context  of  the  current  focus  redirects  the  planners  onto 
a  new  focus.  For  example,  the  planners  were  discussing  what  the  commander 
would  accept  as  a  suitable  obstacle  plan  when  the  question  'Where  do  you 
want  my  company  to  be?"  redirected  the  objective  to  routes  and  position 
Issues. 

New  Information  (NI):  Transitions  which  occur  because  new  information  that 
is  introduced  to  the  planners  redirects  their  focus.  For  example,  the 
planners  were  discussing  obstacle  and  engineering  issues  when  the  new 
information  "I  might  be  able  to  get  you  a  chemical  release"  was  presented 
which  redirected  the  objective  to  artillery  issues. 

Old  Information  (01):  Transitions  which  occur  because  old  information  that 
is  reintroduced  to  the  planners  redirects  their  focus. 

Natural  Resolution  (NR):  Transitions  which  occur  because  an  issue  has  been 
taken  as  close  to  completion  as  is  possible  at  a  particular  time  in  the 
planning  process  and  therefore  a  natural  transition  is  made  to  another 
focus.  A  decision  could  very  likely  lead  to  a  natural  resolution.  For 
example,  the  planners  were  discussing  routes  and  positions  when  they 
resolved  the  issue  ("If  you  think  [these  roads]  will  be  good  then  I'll  just 
use  this  one  right  here.")  and  then  proceeded  to  a  friendly  strategy 
objective. 

Return  to  Previous  Context  (RP):  A  transition  which  occurs  as  a  result  of 
an  individual  Intentionally  redirecting  the  planning  group  to  the  focus  of  a 
previously  addressed  issue.  For  example,  the  planners  were  discussing  ob¬ 
stacles,  when  an  out-of-context  question  redirected  them  to  friendly  strat¬ 
egy.  Shortly  thereafter,  the  S3  tells  the  distracted  individuals  to  "... 
look  on  those  overlays.  Checkpoint  60  on  the  route..."  and  then  redirects 
the  discussion  back  to  obstacles  with  'What  I'm  doing  now,  I'm  trying  to 
figure  out  where  to  put  obstacles..." 
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Time  Break  or  Change  of  Circumstances  (XXX):  A  transition  which  is  the  re¬ 
sult  of  a  break  in  action,  taping,  etc*,  e.g*,  a  break  for  lunch. 


Forced  Change  (FC):  Any  transition  which  occurred  because  an  individual 
abruptly  forced  the  planning  focus  to  another  area*  This  resulted  when 
someone  of  "tank"  redirected  the  planning,  for  instance,  for  a  briefing. 

For  example,  the  planners  were  discussing  routes  and  positions  when  the 
commander  appeared  and  asked  "Where  is  my  main  defense  area  now?"  which  re¬ 
directed  the  objective  to  friendly  strategy. 

Option  Generation  (OG):  A  transition  which  occurred  as  a  result  of  an  in¬ 
context  generation  of  a  new  option  which  redirected  the  planning  to  another 
focus.  For  example,  the  planners  were  discussing  artillery  support  to  help 
blow  bridges  when  the  option  "Options  sheet,  don’t  blow  all  the  bridges"  was 
generated  which  redirected  the  objective  to  a  discussion  of  the  use  of 
obstacles. 

Table  8  shows  the  frequencies  and  relative  percentages  for  these  eleven 
categories  of  transitions  between  objective  segments. 

Out-of-context  questions  and  out-of-context  appraisals  accounted  for  nearly 
one-third  of  all  transitions  while  in-context  appraisals  and  questions  ac¬ 
counted  for  20  percent.  The  category  of  Old  Information  was  initially  hypothe¬ 
sized  as  being  a  potential  reason  for  transitions  to  take  place,  however,  it 
was  not  responsible  for  any.  Only  eight  percent  of  the  transitions  were  the 
result  of  natural  resolutions  (some  of  which  were  decisions). 


Table  8 

Nature  and  Frequencies  of  Transitions 


Nature  of  Transition 

Frequency 

Percent 

Out-of-Context  Questions 

19 

29.7X 

In-Context  Appraisal 

12 

18.8 

Time  Break  or  Change  of  Circumstance 

7 

10.9 

New  Information 

6 

9.4 

Natural  Resolution 

5 

7.8 

Return  to  Previous  Context 

5 

7.8 

Forced  Change 

5 

7.8 

In-Context  Questions 

2 

3.1 

Out-of-Context  Appraisal 

2 

3.1 

Option  Generation 

1 

1.6 

Old  Information 

_0 

0.0 

Total 

64 

100. OX 
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One  implication  of  this  finding  is  that  in  a  group  decision  environment  such 
as  this  one  there  is  a  great  deal  of  accidental  shifting  from  one  focus  to 
another.  This  process  is  one  that  mist  be  actively  managed  in  order  to  keep  it 
from  becoming  chaotic* 

Progressive  deepening.  Progressive  deepening  is  a  model  of  deliberation  by 
which  an  individual  evaluates  an  option  or  idea  by  gradually  examining  deeper 
and  deeper  “branches"  of  the  idea  for  workability  or  flaws.  Eventually  a  point 
is  reached  where  the  idea  is  either  accepted,  rejected,  or  left  hanging  due  to 
some  distraction.  If  it  is  rejected,  the  decision  maker  either  moves  on  to  a 
totally  different  option  or  goes  back  up  the  deepening  chain  to  a  point  (theo¬ 
retically)  above  the  source  of  the  flaw  and  then  follows  another  branch,  once 
again  testing  its  merits  all  along  the  way* 

The  deliberation  processes  of  four  separate  segments  of  planning  discussions 
were  charted  to  see  if  the  Arny  planners  demonstrated  progressive  deepening* 

The  charts  for  these  four  segments  are  included  in  Figures  1,  2,  3,  and  4.  The 
respective  transcripts  which  correspond  to  the  charts  are  included  as  supple¬ 
ments  to  each  figure.  Table  9  which  follows  Figure  1,  is  a  narrative  descrip¬ 
tion  of  the  steps  contained  in  that  chart  and  is  intended  to  give  an  example  of 
how  to  read  the  charts. 

The  first  three  diagrams  are  charts  of  the  continuing  deepening  of  the  same 
topic  through  three  noncontiguous  segments  of  the  planning  process.  The  first 
and  second  segment  were  separated  by  approximately  30-45  minutes,  while  the 
second  and  third  were  separated  by  an  additional  two  and  a  half  hours. 


MiU«*  |  J*»*j  |  mt 

Lut  flit  [  r4ti  Uii 


(ltani  don  M  I  tk*t 


SUPPLEMENT  1 

TRANSCRIPT  ACCOMPANYING  FIGURE  1 
REFERENCE  UNIT  #25 

Approx*  30  min*  into  planning  session* 


X:  I  think  that's  a  great  idea,  why  not  use  an  already  made  obstacle.  They 
would  try  to  avoid  this,  this  is  where  we  need  to  stop  them  from  using,  stop 
them  from  using  that  road  right  there.  Only  way  to  stop  them  is  to  blow  this 
mine  here,  that  bridge  there.  See  if  you  crater  that  road  all  they  do  is  they 
just  pull  off  the  road  go  in  the  woods  and  come  back  on  the  road.  That  might 
work,  I'd  say  that  would  be  a  good  idea  there.  Mines  on  both  sides.  Well,  ok, 
so  we  crater  it,  we  put  mines  on  both  sides,  so  they  pull  off  the  road  cause 
they  can't  use  it,  they  see  mines,  so  they're  backing  up,  that '6  going  to  slow 
them  way  down,  if  they  see  the  mines  in  time  what's  keeping  them  from  just 
swinging  out  and  just  getting  back  on  the  road,  that's  what  I  would  do,  that'B 
the  only  thing.  Which  would  be  a  better  place  to  crater  it?  What's  that  do  for 
U6 ,  how  does  that  help  us. 

Table  9  is  a  narrative  description  of  Figure  1  and  is  included  as  an  exam¬ 
ple  of  how  the  figures  are  to  be  interpreted. 
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Table  9 


Narrative  Description  of  Progressive  Deepening,  Figure  1 


Step  Step  Explanation _ 

1.  The  planner  examines  the  map  and  notes  that  they  need  to  flatten 
the  east  side  of  the  railroad  track  and  observes  that... 

2.  By  doing  this  they  will  make  use  of  an  already  made  obstacle* •• 

3.  This  will  cause  the  enemy  to  try  to  avoid  it. 

4.  The  planner  looks  further  eastward  on  the  map.  He  then  notes  that 
they  will  need  to  deny  the  enemy  the  use  of  “this"  particular 
road. • . 

5.  One  way  to  help  accomplish  this  is  to  blow  or  mine  a  particular 
bridge. • . 

6.  The  enemy  can  also  be  denied  the  use  of  “this"  road  by... 

7.  Cratering  the  road... 

8.  But  the  enemy  will  just  pull  off  the  road,  go  around  the  craters, 

pull  back  on  and  continue... 

9.  We  could  mine  both  sides  of  the  road... 

10.  We  could  crater  the  road  and  mine  both  sides,  then... 

11.  The  enemy  will  pull  off  because  of  the  craters... 

12.  They  will  see  the  mines... 

13.  They  will  have  to  back  up,  resulting  in... 

14.  The  enemy  being  slowed  down. 

15.  But  what  if  the  enemy  sees  the  mines  in  time... 

16.  He  will  swing  out  early  and  go  around  the  mines  as  well  and  then 
get  back  on  the  road  and  continue... 

17.  The  planner  indicates  that  he  would  do  it  that  way. 

16.  This  problem  causes  a  brief  pause. 

19.  If  we  crater  the  roads... 

20.  Where  is  the  best  place?..* 

21.  How  will  cratering  the  roads  help  us?... 

22.  It  will  deny  the  enemy  the  use  of  the  road,  therefore... 

23.  The  enemy  won’t  be  able  to  get  through  to  "this"  battle  position. 

24.  This  will  be  O.K.  if  we  take  the  road  out  here. 
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SUPPLEMENT  2 

TRANSCRIPT  ACCOMPANYING  FIGURE  2 
REFERENCE  UNITS  II  45-48 
Approx.  45-60  minutes  into  planning  session 


Y;  What  I  want  is  a  crater,  it  doesn’t  have  to  be  a  mine  field,  I  just  want  a 
crater,  I'm  just  putting  a  symbol  there  to  show  that  there's  some  demolitions 
there.  I  guess  I  could  just  go  with  the  X,  which  is  what  I  was  told.  Naw, 
just  as  long  as  the  bridge  is  taken  out,  that's  what  I  want  done. 

Z;  Bob,  why  don't  you  do  that  intersections  up  there? 

Y:  What's  that? 

Z:  On  phase  line  Idaho,  up  in  the  north,  that  big  intersection,  crater  that 
mother. 

Y:  What  will  that  make  them  do  other  than  pull  off  the  road? 

Z:  We're  going  to  get  artillery  on  them  in  there,  cause  they're  not  going  to 
be  able  to  do  very  much  in  the  woods,  but  nobody  can  observe  the  artillery 
there  though. 

Y;  I  see  what  you're  saying,  we  thought  about  that  one,  but... 

Z:  or  place  if  further  to  the  west  down  in  the  dip  or  in  that  open  area  some¬ 
where.  What  do  we  have  engineer  for  assets,  how  much  mines  have  we  used  and 
what  we've  got  now? 
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SUPPLEMENT  3 

TRANSCRIPT  ACCOMPANYING  FIGURE  3 
REFERENCE  UNITS  148-158 
Approx.  2-1/2  to  3  hours  into  Planning  Session 


W:  yeah,  that’s  what  we  were  talking  set  up  ambushes  in  the  woods  and  stuff, 
then  thought  about  blowing  like  some  of  those  major  road  intersections,  like  if 
he  does  come  through  here  at  least  force  them  into  the  trails  to  slow  them 
down. 

Y:  We  thought  about,  we  talked  about  it,  the  reason  I  didn’t  put  anything  is 
that's  another  mission  for  engineers  that  are  already  strung  out,  and  realis¬ 
tically  with  no  one  there,  if  I  did  blow  them  all  they  would  do  is  pull  off  the 
road,  get  back  on  it  and  just  keep  on  trucking.  Ok,  we  crater  that  intersec¬ 
tion,  they  come  to  it,  they  get  off  the  road  go  around  it,  and  get  right  back 
on  again. 

W:  Mine  the  sides. 

Y:  They'll  just  go  around  the  mines.  Ok,  how  about  a  spot  for  a  FASCAM  then, 
would  that  make  sense,  how  about  either  side  of  the  roads,  in  the  trees. 

Z:  Trees  are  not  good. 

Y:  No,  ok. 

W:  See  the  whole  purpose  of  doing  that  is  so  they  don't  catch  the  redball  and 
here  again  you're  displacing  you  don't  get  into  a  road  rally. 

Z:  Soft  terrain  is  best,  yeah,  I  can  throw  it  in  the  trees. 

Y:  Ok,  then  how  about  in  the  moors,  we  got  an  open  area,  how  about  a  FASCAM  on 
this  side,  maybe  a  mine  field  on  this  side  then,  crater  the  road,  would  that  be 
a  good  spot  then?  Ideally  would  be  maybe  a  FASCAM  here  and  a  FASCAM  on  that 
side. 

Z:  Cover  your  flank  or... 

Ys  What  we're  trying  to  do  is  to  deny  them  this  road  if  they  decide  to  use  it. 
If  we  drop  a  FASCAM  here  and  here  and  crater  the  road  that's  a  major  obstacle 
now,  now  they  have  to  swing  away,  either  down  this  way  or  way  up  this  way  if  we 
do  that. 

W:  Right  in  there,  there's  enough  of  a  clearing  to  put  about  a  1000  meters, 
they  run  up  to  the  crater  and  start  playing  around  in  the  FASCAM,  you  bloody 
their  nose,  you  now  know  that  they  are  there,  and  then  he  gets  out. 

Y:  Ok,  I  give  that  to  Charlie  then. 

X:  All  we  need  is  a  squad  and  then  hope  that  anybody  filing  out  will  try  to  go 
down  to  the  south. 

V:  Ok,  lets'  try  that  and  see  how  that  works. 
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X:  Looks  like  ambush  all  day  and  if  it  slows  them  down  10  or  15  min  it  gives 
you  that  much  more  time  back  here  to  reset.  If  they’re  all  coming  to  the  north 
it  will  give  delta  some  more  time  to  push  up  to  the  north. 

Y:  Sound  great  I  like  that,  that’s  what  we  need  to  do.  Just  don’t  give  them 
the  road,  that’s  why  I  was  trying  to  mine  these  here. 
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Figure  4,  Progressive  deepening  diagram  for  unit  $26. 
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SUPPLEMENT  4 

TRANSCRIPT  ACCOMPANYING  FIGURE  4 
REFERENCE  SECTION  if  26 
Approx.  35  sin.  into  planning  session. 


X:  It  keeps  them  from  using  that  road  to  come  down  on  this  battle  position  I 
agree  with  that,  if  you  take  it  out  here.  Ok,  I'd  say  rather  than  sitting  in 
here  I  think  they  should  be  up  here,  use  this  fire  zone  here.  I  guess  I  basi¬ 
cally  agree  with  what  the  Col.  said,  if  it  works  right,  cause  the  thing  that  is 
important  in  this  is  routes  in  and  out  of  these  positions,  if  you  don't  have 
them,  then  you  die  with  that.  Let's  use  the  road,  why  not,  use  the  opposite 
side  of  the  road,  around  the  edge  of  the  town.  Ok,  so  I've  got  an  M21,  do  I 
want  to  put  them  there,  why  am  I  putting  them  there?  Yes,  it's  kind  of  a  big 
ridge  is  what  it  is,  I  just  can't  see  them  going  into  here.  We  want  then  to  go 
into  there  but  will  they  go  into  there*  I  agree  with  these  positions,  these 
make  sense  because  you  take  that  dozer  and  just  level  that  out.  I  think  this 
is  where  our  main  attempt  should  be  right  in  there. 
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The  diagrams  indicate  that  progressive  deepening  was  taking  place.  The 
planners  worked  their  way  down  branches,  evaluated  as  they  went,  occasionally 
jumped  back  up  to  earlier  nodes  and  them  branched  off  in  slightly  different 
directions.  A  variation  on  this  pattern  took  place  when  a  break  in  time  was 
involved  and  the  planners  returned  to  a  partially  deepened  topic  that  had  been 
addressed  earlier.  In  these  cases  they  generally  started  on  a  higher  node  than 
where  they  previously  left  off  and  sometimes  jumped  around  through  branches 
already  covered.  Eventually  though,  they  settled  down  to  a  more  consistent 
deepening  pattern  and  took  it  to  a  point  farther  along  than  they  had  previously 
reached. 

The  concept  of  progressive  deepening  was  introduced  by  de  Groot  (1978)  in 
his  protocol  analysis  of  chess  grandmasters.  It  is  a  very  effective  strategy 
for  pursuing  the  implications  of  an  option.  It  is  contrasted  to  a  broad-based 
search  whereby  a  large  number  of  options  are  scanned  to  a  shallow  depth.  By 
relying  on  progressive  deepening,  the  battle  managers  were  using  a  sophisti¬ 
cated  strategy  that  placed  additional  demands  on  them  because  it  requires  the 
ability  to  recall  the  way  the  line  of  reasoning  was  developed,  and  to  judge  the 
best  lines  for  continued  evaluation. 

By  its  very  nature,  progressive  deepening' s  free-flowing,  informal  reasoning 
allow  the  group  decision  processes  to  be  accidently  sidetracked.  However,  we 
have  also  seen  that  with  the  use  of  a  structuring  aid,  such  as  a  synchroniza¬ 
tion  matrix,  this  shifting  of  focus  is  minimized. 

Assessment  of  CDM 


CDM  applications  in  military  settings.  CDM  can  be  effectively  used  in  a 
military  setting.  The  Ft.  Hood  exercise  demonstrated  that  the  method  could  be 
applied  in  dynamic  battalion  pre-battle  planning. 

CDM  originally  was  designed  as  a  post-event  interview  to  assist  researchers 
in  extracting  information  from  participants  about  their  decision-making  proc¬ 
esses.  It  has  been  demonstrated  to  be  flexible  enough  to  work  at  least  as 
well,  if  not  better,  in  situations  where  the  researchers  are  present  during  the 
event.  In  these  circumstances,  the  interviewers  generally  take  notes,  observe, 
and,  when  there  are  breaks  in  the  action,  interview  key  participants.  This 
latter  variation  of  CDM  is  the  method  employed  during  the  Ft.  Leavenworth  and 
Ft.  Riley  data  collection  trips.  Another  iteration  of  the  method  was  demon¬ 
strated  during  the  Ft.  Hood  exercise  using  adjunct  data  (from  audio  tapes)  to 
substitute  for  many  of  the  CDM  probes  and  potentially  allowing  a  more  effective 
set  of  probes.  In  this  case,  the  planners  were  involved  in  group  decision 
making  involving  large  amounts  of  verbal  communication  at  a  central  location. 
During  the  course  of  the  interviewing  for  this  particular  exercise  it  became 
evident  that  answers  to  the  interview  questions  were  already  available  in  the 
content  of  the  discussions  taking  place  in  real-time.  That  is,  we  appeared  to 
be  wasting  time  asking  the  S3  about  which  options  he  considered  at  a  given 
decision  point,  since  the  answer  was  clear  from  the  discussion  we  had  taped. 

So,  even  though  the  participants  were  very  cooperative  with  the  interviewers, 
the  process  was  deemed  to  be  redundant.  The  answers  to  CUM  probes  were  ex¬ 
tracted  from  the  real-time  transcripts.  This  method  tended  to  preclude  the 
problem  of  participant  recall  which  arises  with  any  post-interview  method. 
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The  results  indicate  that  variations  of  CDM  can  be  applied  in  quite  diverse 
environments  including:  a)  non-observation  with  post-event  interviews;  b) 
direct  observation  with  during-event  and  post-event  interviews;  and  c)  direct 
observation  and  audio  taping,  with  data  extraction  applied  to  the  real-time 
transcripts.  Here,  also,  direct  observation  clarified  missing  visual  informa¬ 
tion. 

Modeling  group  decision  making.  CDM  has  previously  been  applied  to  individ¬ 
ual  decisions.  The  Ft.  Hood  scenario  allowed  the  researchers  to  track,  a  group 
decision  process.  The  nature  of  the  planning  exercise  permitted  this.  The 
result  was  that  this  type  of  protocol  analysis  allowed  us  to  "map'*  the  entire 
five  hours  of  the  planning  process  into  the  categories  previously  discussed. 

The  "map"  itself  is  a  fairly  intimidating  22  foot  long  sheet  of  computer  paper 
(see  excerpt,  Appendix  C).  However  it  allows  one  to  conceptualise  the  proc¬ 
esses  taking  place  in  order  to  facilitate  other  analyses. 

Discussion  and  Conclusions 
Applications  to  Natural  Settings 

The  CDM  was  applied  in  three  separate  military  settings.  Each  of  these 
involved  a  command  post  exercise  or  field  training  exercise  in  which  a  command 
post  staff  operated  on  dynamic  information  to  develop  plans  under  near  real 
time  constraints.  The  last  of  these  exercises,  a  computer  driven  battalion 
level  exercise  at  Fort  Hood,  Texas,  offered  the  richest  source  of  data.  It  was 
decided  to  focus  the  decision  analysis  efforts  entirely  on  these  data. 

Management  of  the  Planning  Process 

There  is  a  need  for  military  planners  to  be  trained  in  the  management  of  the 
planning  process.  Military  planning  is  generally  viewed  as  the  artful  manipu¬ 
lation  of  critical  elements  such  as  strategy,  intelligence,  tactics,  logistics, 
fire  and  air  support,  planning  for  Immediate  versus  future  battles  and  so  on. 
Indeed,  when  military  planning  was  chosen  as  the  domain  for  this  project,  we 
were  very  impressed  with  the  amount  of  specific  domain  knowledge  required  Just 
to  follow  the  planning  discussions  as  they  were  taking  place.  However,  the 
management  of  the  process  would  appear  to  be  as  critical  as  the  actual  planning 
itself.  Military  planners  are  specifically  trained  to  work  with  the  domain 
specific  knowledge  but  receive  relatively  little  training  in  the  management  of 
the  process. 

Our  analysis  showed  64  major  transitions  between  objectives  of  discussion 
during  the  five-hour  planning  session.  This  indicates  that  the  planners  were 
switching  objectives  about  every  five  minutes.  This  does  not  prove  that  the 
process  was  managed  poorly,  but  it  does  emphasize  the  need  for  management. 

During  these  64  discussions,  only  one  objective  out  of  12  (movement  of  friendly 
troopB  from  a  previous  battle  sector  to  the  current  one)  was  addressed  and 
followed  straight  through,  uninterrupted,  to  resolution. 

We  were  surprised  that  332  of  the  transitions  were  essentially  "discon¬ 
nects",  i.e.,  out-of-context  factors  such  as  outside  interruptions  that  moved 
the  planning  discussions  from  one  topic  to  a  totally  unrelated  one.  Often 
after  these  transitions,  the  discussions  never  returned  to  the  specific  focus 
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of  the  earlier  topic.  Another  important  aspect  of  these  transitions  was  that 
they  often  occurred  without  any  apparent  concern  on  the  part  of  the  planners. 
They  did  not  seem  to  notice  that  they  had  left  one  topic  behind  and  were  now  on 
a  completely  unrelated  one. 

In  comparison,  a  little  over  20%  of  the  transitions  occurred  as  a  result  of 
in-context  factors.  Once  again,  these  transitions  were  often  extremely  subtle. 
The  planners,  starting  from  within  the  current  context  often  completely  re¬ 
directed  themselves  to  a  totally  unrelated  topic,  without  verbalizing  any 
awareness  that  a  transition  had  taken  place. 

In  addition  to  transitions  between  topics,  the  development  of  the  plan  in¬ 
volved  processing  large  amounts  of  information  (164  separate  pieces)  and  the 
generation  and  (sometimes)  serial  evaluation  of  numerous  options  (102).  Track¬ 
ing  this  information  and  these  options,  in  addition  to  monitoring  the  topic 
transitions,  places  a  real  burden  on  the  management  of  the  process. 

Group  development  of  a  plan  under  time  pressure  is  beset  by  confusion,  over¬ 
loading  and  chaos  by  its  very  nature.  This  same  confusion,  overloading  and 
chaos  makes  managing  the  process  that  ouch  more  important  and  difficult.  It  is 
hard  to  manage  the  planning  discussion  while  you  are  participating  in  it. 

We  will  expand  on  these  ideas  in  following  sections  and  present  recommenda¬ 
tions  for  training  and  decision  support  systems. 

Information  Flow  and  Access 


During  the  interviews  with  the  planners,  one  point  was  consistently  dis¬ 
cussed  —  the  need  for  more  and  better  information.  Although  many  types  of 
information  were  mentioned,  the  one  that  was  touched  upon  by  everyone  was  for 
more  complete  and  accurate  enemy  intelligence.  Other  popular  topics  were  the 
need  for  planners  to  actively  seek  out  the  information  they  lacked,  to  keep 
track  of  this  Information  and  to  know  when  to  apply  it. 

» 

There  were  two  main  points  of  contradiction  between  the  interview  comments 
and  the  planners’  actual  performance  as  observed.  The  first  was  that  intelli¬ 
gence  about  enemy  actions  was  repeatedly  identified  as  a  primary  need  yet  the 
S3  planners  never  used  S2  (Intelligence)  as  a  primary  source  of  information. 
There  were  seven  main  sources  of  information  identified  during  the  five  hours 
of  planning  (see  Table  8).  These  ranged  from  individuals  present  In  the  plan¬ 
ning  room  to  the  Commander  and  the  brigade  headquarters.  S2  was  not  one  of 
these  sources. 

The  failure  to  use  the  S2  may  have  been  the  result  of  the  second  contradic¬ 
tion  that  we  observed:  the  planners  did  not  actively  seek  information  they 
needed.  If  the  material  was  not  in  the  immediate  area  (i.e.,  in  the  planning 
room)  and  easily  accessed,  they  did  not  go  looking  for  it.  In  fact,  93%  of  the 
information  was  from  three  sources  which  were  right  there  with  the  planners; 
individuals,  the  map,  and  the  plan  (overlays,  etc.).  The  remaining  seven  per¬ 
cent  was  from  the  four  sources  which  were  not  as  easily  accessed:  the  com¬ 
mander,  brigade  headquarters,  the  operations  order,  and  Soviet  doctrine  materi¬ 
als.  Indeed,  when  the  operations  order  and  Soviet  doctrine  were  sources  of 
information  it  was  because  someone  in  the  immediate  area  happened  to  be  able  to 
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recall  the  information  from  memory,  i.e.,  they  did  not  go  ’’look  it  up."  The 
last  update  from  the  S2  was  approximately  two  to  three  hours  prior  to  the  ana¬ 
lyzed  five  hour  planning  segment.  Following  this  briefing,  the  S2  worked  ex¬ 
clusively  in  his  own  section  and  no  one  in  the  S3  section  solicited  update 
information  from  the  S2.  The  S2  vehicle  was  only  about  25  feet  from  the  pri¬ 
mary  S3  planning  area  but  they  were  not  in  direct  line-of-sight,  thus  making 
the  S2  area  somewhat  less  accessible  to  the  S3  section. 

Another  interesting  point  concerns  passively  acquired  versus  actively  ac¬ 
quired  information.  Passively  acquired  information  outweighed  actively  ac¬ 
quired  information  by  greater  than  2  to  1.  Of  the  actively  acquired  informa¬ 
tion,  we  found  that  most  of  it  was  still  from  the  same  three  immediately 
accessible  sources  (82%).  We  were  surprised  at  this  result  primarily  because 
the  planners'  comments  in  the  interviews  led  us  to  expect  that  they  had  more 
actively  attempted  to  acquire  whatever  information  was  needed. 

In  summary,  the  planners  did  not  use  information  from  52  even  though  they 
repeatedly  stated  that  they  needed  more  enemy  intelligence.  Also,  if  the  in¬ 
formation  was  not  very  easily  accessible,  (i.e.,  in  the  room)  they  more  than 
likely  did  not  track  it  down. 

The  implication  is  that  information  may  not  flow  unless  it  is  already  close 
at  hand.  This  highlights  the  need  for  decision  aids  and  support  systems  that 
will  help  make  information  very  easily  accessible.  Training  is  needed  to  in¬ 
crease  the  awareness  as  to  the  necessity  for  more  active  retrieval  of  these 
materials. 

Decision-Making  Strategies 

The  classical,  prescriptive  models  of  decision  making  typically  indicate 
that  decisions  should  be  made  through  a  deliberate  process  where  as  much  infor¬ 
mation  as  possible  is  compiled  about  available  options.  These  options  are  then 
deliberated  concurrently  end  a  final  decision  is  made  based  upon  a  multi-at¬ 
tribute  weighting  which  allows  a  determination  of  the  strongest  alternative. 
This  becomes  the  chosen  option.  In  fact,  Army  doctrine  is  consistent  with  pre¬ 
scriptive  decision  models  suggesting  military  planners  should  generate  a  set  of 
options  and  select  the  best.  The  main  question  that  arises  is  whether  or  not 
this  concurrent  deliberation  model  is  relevant  to  natural  decision  making  sit¬ 
uations. 

We  have  shown  (Klein,  Calderwood,  &  CHnton-Cirocco,  1986;  Klein,  in  press) 
that  "experts"  in  natural  decision-making  settings  operating  under  various 
levels  of  time  pressure  and  stress  do  not  follow  the  classical  decision  proc¬ 
ess.  These  individuals  have  been  shown  to  approach  the  decision-making  process 
by  addressing  one  single  option  at  a  time.  Often  this  involves  making  an  auto¬ 
matic  decision,  with  little  if  any  deliberation.  When  multiple  option  delib¬ 
eration  does  occurs,  it  tends  to  be  along  serial,  rather  than  concurrent, 
lines.  In  this  case  the  decision  maker  considers  one  option  at  a  time  and 
either  accepts  it  as  a  workable  solution,  or  rejectB  it  and  then  moves  on  to  a 
completely  different  option  and  repeats  the  process.  These  decisions,  involv¬ 
ing  automatic  and/or  serial  consideration  of  options  have  been  labeled  Recogni¬ 
tion-Primed  Decisions  (RPDs). 
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As  mentioned,  In  our  previous  studies  involving  Individual  decisions  with 
experienced  individuals  working  in  familiar  domains,  the  majority  of  the  deci¬ 
sions  were  RPDs.  This  study  differed  in  that  it  was  a  group  decision  setting 
with  a  broad  mix  of  levels  and  types  of  experience  and  expertise*  However,  we 
found  96  percent  of  the  identified  decisions  (26  out  of  27)  were  RPDs  where 
there  was  no  concurrent  deliberation.  That  is,  multi-attribute  weighting  of 
options,  decision  analysis  and/or  Bayesian  strategies  did  not  come  into  play. 
This  represents  an  even  higher  percentage  of  RPDs  in  this  group  decision  set¬ 
ting  than  in  previously  studied  individual  decision-making  settings. 

The  planners  we  studied  consistently  used  RPD,  serial  decision  strategies, 
much  like  other  experienced  decision  makers  we  have  observed.  This  involved 
finding  options  that  could  potentially  satisfy  the  requirements  of  the  situa¬ 
tion  and  "playing  out"  the  option  in  their  minds  or  on  a  map  through  progres¬ 
sive  deepening.  These  individuals  did  not  concurrently  deliberate  options  even 
though  Army  doctrinal  writings  suggest  it  should  be  done. 

Military  decision  training  has  traditionally  been  based  upon  Multi  Attribute 
Utility  (MAU)  analysis,  decision  analysis,  or  Bayesian  approaches.  These  ap¬ 
proaches  are  inconsistent  with  the  decision  strategies  that  are  being  employed 
by  these  military  planners  (or  any  other  job  incumbent  we  have  observed).  If 
one  accepts  the  assumption  that  experts  are  deemed  such  because  they  generally 
make  correct  and  timely  decisions,  then  a  logical  approach  to  training  might  be 
to  capture  the  salient  aspects  of  their  decision  strategies  by  knowledge  elici¬ 
tation  and  then  to  convey  these  same  strategies  to  individuals  with  less  exper¬ 
tise.  The  results  reported  in  this  paper  suggest  that  the  traditional  training 
of  MAU,  Bayesian  and  Decision  Analysis  techniques  is  an  unnatural  approach.  As 
such,  it  could  potentially  interfere  with  the  learning  process  and  reduce  the 
credibility  of  the  training,  resulting  in  an  inefficient  use  of  valuable  train¬ 
ing  time. 

Evolution  of  Method  of  Data  Collection  and  Analysis 


Initial  projects  involved  conducting  CDM  interviews  after  the  fact  (Klein, 
Calderwood,  &  Clinton-Cirocco,  1986).  In  these  cases  the  participants  would 
reconstruct  an  important  event  and  identify  critical  points.  The  researchers 
would  then  interview  the  individuals  about  the  critical  decisions  and  cues. 

A  later  variation  of  the  CDM  allowed  the  researchera  to  be  on  site  during 
the  course  of  the  major  events  (Taynor,  Klein,  &  Thordaen,  1987;  Bretovic, 
Klein,  &  Thordeen,  1987).  This  permitted  the  researchers  to  conduct  the  inter¬ 
views  with  the  participants  either  afterwards  or  during  breaks  in  the  incident. 
The  researchers  were  able  to  use  direct  observation  and  their  notes  of  the 
event  to  verify  aspects  of  the  recall  of  the  interviewees. 

These  two  early  approaches  were  very  effective  in  providing  data  concerning 
critical  decisions  and  the  cues  attended  to  by  the  decision  makers.  This  in¬ 
cluded  much  detailed  information  about  the  decisions  such  as  deliberation 
strategies,  time  pressure  and  options.  These  approaches  were  very  effective  in 
capturing  the  occurrences  directly  preceding,  during  and  immediately  following 
decisions.  They  did  not  attempt,  however,  to  capture  the  intervals  between 
decisions. 
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The  data  collection  approach  initially  envisioned  for  this  study  was  to  exam¬ 
ine  the  military  planning  process  from  two  perspectives  simultaneously:  S3 
(Planning)  and  S2  (Intelligence).  Two  researchers  were  to  observe  (one  in  each 
section)  the  development  of  a  military  plan.  Critical  incidents  and  time 
frames  were  to  be  used  as  anchors  for  focusing  the  CDM  interviews.  In  the 
observation  at  Ft.  Leavenworth,  Ft.  Riley  and  the  initial  periods  at  Ft.  Hood 
this  approach  was  followed. 

During  the  course  of  the  initial  interviews  at  Ft.  Hood  the  observers  noted 
that  the  responses  often  covered  the  same  ground  the  planners  had  just  openly 
discussed  in  the  actual  planning  session.  Based  upon  this  discovery,  it  was 
decided  to  audio  tape  record  the  actual  planning  session  itself  and  to  use  the 
Interviews  to  address  material  which  could  not  be  picked  up  from  the  planning 
discussions.  The  resulting  transcribed  data  were  based  directly  upon  the  ac¬ 
tual  event  and  did  not  need  to  rely  on  the  participants’  and  researchers'  re¬ 
call.  The  data  could  then  be  analyzed  by  mapping  the  development  of  the  plan 
and  in  essence  extracting  many  of  the  interview  "responses"  directly  from  the 
transcripts. 

This  application  of  CDM  resulted  in  a  data  analysis  that  was  primarily 
driven  by  the  progressive  deepening  model  of  decision  making  (based  on  the 
entire  flow  of  the  planning  process).  Thi6  process  generated  an  extremely  rich 
data  set.  However,  there  were  several  weaknesses  in  this  approach.  The  first 
was  that  less  effort  was  given  to  analyzing  the  CDM  interviews  than  to  analyz¬ 
ing  the  transcripts  of  the  planning  session.  The  second  weakness  was  that  we 
used  our  usual  format  for  the  CDM  interviews  rather  than  shifting  focus  to  use 
a  more  valuable  set  of  probes  (e.g.,  probes  getting  specifically  at  training 
issues).  It  was  only  afterwards  when  we  reviewed  the  transcripts  that  we  real¬ 
ized  that  the  CDM  interviews  were  redundant  with  the  tape  recordings.  A  third 
weakness  was  that  the  transcripts  did  not  reveal  some  subtle  aspects  of  deci¬ 
sion  strategies  and  situation  assessment,  aspects  that  could  have  been  elicited 
by  the  CDM  if  it  had  been  properly  coordinated  with  the  transcript  data. 

We  used  several  different  methods  for  the  final  analysis  of  the  data  for 
this  project.  The  first  was  a  systematic  mapping  of  the  content  analysis 
(Appendix  C).  This  mapping  allowed  us  to  systematically  track  the  process  as 
it  deepened  and  as  it  jumped  around  during  the  transitions.  The  second  method 
involved  extracting  the  timing  and  frequency  or  various  types  of  planning  ac¬ 
tivities  directly  from  the  transcripts.  The  third  method  was  the  development 
of  the  progressive  deepening  charts.  This  is  an  approach  to  protocol  analysis 
we  hove  not  seen  used  before,  where  the  charts  were  developed  directly  from  the 
transcripts  of  complex,  real  world  events.  De  Groot  (1978)  and  Newell  and 
Simon  (1972)  have  obtained  such  data  from  artificial  protocol  gathering  "think 
aloud"  studies.  However,  because  we  were  recording  a  group  decisionnoaking 
session  we  could  be  less  intrusive.  Finally,  the  fourth  method  employed  was 
the  original  CDM. 

All  four  of  these  analysis  methods  were  beneficial  in  helping  to  understand 
the  group  decision  processes.  However,  it  is  best  to  consider  the  first  three 
as  separate  tools  to  be  applied  alongside  the  CDM.  Extracting  information 
after-the-fact  from  the  transcripts  provided  some  of  the  information  normally 
gained  by  the  CDM  interviews.  At  the  same  time  strict  cognitive  processing  was 
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not  captured  by  this  method#  That  is,  we  were  able  to  identify  options  gener¬ 
ated,  the  overall  manner  in  which  deliberation  took  place  (serial  versus  con¬ 
current),  the  amount  of  time  of  deliberation,  available  Information,  some 
critical  cues,  decision  points,  types  of  RPDs,  etc.  However,  certain  material 
normally  gained  from  the  direct  COM  interviews  was  not  captured  by  the  other 
methods.  This  included  much  of  the  data  from  the  "higher  order"  probes  such 
as:  How  would  the  individuals  have  handled  a  situation  at  an  earlier,  less 
experienced  period  in  their  lives?  What  decisions  were  considered  "critical 
decisions"  by  the  individual?  Why  were  these  decisions  considered  critical? 

What  were  the  specific  critical  cues  attended  to  by  the  individual?  In  future 
studies  we  would  reserve  the  CDM  interviews  for  these  types  of  probes  and  col¬ 
lect  the  decision  strategy  data  from  the  tapes.  We  feel  that  combining  the  use 
of  these  methods  would  be  a  good  way  to  strengthen  the  process  of  studying 
complex  situations. 

Implications  for  Decision  Support  Systems  and  Decision  Aids 

As  mentioned  above,  MAU,  Decision  Analysis  and  Baysian  approaches  were  gen¬ 
erally  not  being  used  by  the  planners  during  the  planning.  If  these  planners 
are  typical  of  current  and  future  Arny  planners,  we  question  whether  it  is 
appropriate  or  beneficial  in  this  environment  to  use  any  of  these  approaches  as 
the  foundations  for  decision  support  systems  (DSS)  and  aids. 

The  results  do  indicate  some  key  factors  which  would  be  helpful  to  incorpo¬ 
rate  in  a  DSS.  These  include  information  access,  option  formulation,  and  mem¬ 
ory  and  recall  aids  for  progressively  deepened  parts  of  the  plan.  However, 
these  suggestions  emerge  primarily  from  a  recognitional  model  of  decision  mak¬ 
ing,  as  applied  to  our  data.  The  CDM  and  the  content-analytical  approach  used 
with  the  transcripts  directly  generated  few  recommendations  about  decision 
support  systems,  and  we  do  not  see  them  being  particularly  valuable  for  such 
purposes. 

Information  access.  Information  access  concerns  the  ease  of  retrieval  of 
information  which  is  available  to  the  planners.  The  results  indicated  that  of 
the  sources  of  information  "tapped"  by  the  planners,  three  could  be  classified 
as  extremely  accessible  (individuals,  the  plan,  and  the  map),  two  were  moder¬ 
ately  accessible  (operations  orders  and  Soviet  doctrine)  two  others  were  less 
accessible  (the  Commander  and  brigade  headquarters).  Of  these  sources,  the 
three  extremely  accessible  ones  accounted  for  92.7  percent  of  the  information. 
The  moderately  accessible  sources  accounted  for  4.3  percent  and  the  less  acces¬ 
sible  sources  for  3  percent.  If  the  information  was  not  easily  accessed,  it 
quite  likely  was  not  acquired.  This  point  is  strengthened  when  it  is  noted 
that  Intelligence,  Logistics,  Air  Support*  and  Fire  Support  were  never  specif¬ 
ically  used  as  sources  of  information.  Access  to  this  information  was  not 
extremely  difficult  by  normal  standards;  a  radio  call  or  a  20  meter  walk  across 
the  "compound"  would  have  put  the  planners  in  direct  contact  with  these  sec¬ 
tions*  However,  in  contrast  to  the  individual,  map,  and  plan  sources  these 
were  relatively  unaccessible,  and  were  therefore  not  drawn  upon. 

Designers  have  been  working  on  concepts  to  present  information  in  more  or 
less  detail  for  many  years  now.  Our  observations  have  underscored  the  need  for 
such  efforts.  We  saw  many  cases  where  needed  key  terrain  features  such  as 
bridge  locations  would  not  be  discovered  until  late  in  the  planning.  We  would 
encourage  this  line  of  research  to  continue. 
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An  interactive  local  area  network  tied  into  the  Planning,  Intelligence,  Fire 
Support,  and  Logistics  section  could  provide  easy  access  to  all  about  the  cur¬ 
rent  status  of  different  parts  of  the  plan.  If  this  includes  a  bi-directional, 
non-intrusive  "reminding"  function  which  informs  S3  of  changes  made  by  the 
other  shops  and  tells  the  other  shops  which  parts  of  their  plan  S3  has  "checked 
out,"  it  could  also  serve  to  increase  and  enhance  communications. 

Due  to  the  often  confusing,  chaotic  nature  of  planning,  information  must  be 
extremely  accessible  to  battle  managers,  otherwise  they  do  not  bother  to  ac¬ 
quire  it. 

Option  formulation.  During  the  Ft.  Hood  exercise,  164  discrete  pieces  of 
information  were  mentioned  and  225  instances  of  appraisal  activity  were  re¬ 
corded  in  which  102  identifiable  options  were  generated.  These  options  occa¬ 
sionally  dealt  with  overall  course  of  action  (COA)  issues,  but  in  general, 
focused  on  sub-decisions  concerning  a  given  COA.  Many  of  these  options  were 
quickly  discussed  and  then  rejected,  however  an  equal  number  were  retained  if 
not  for  immediate  incorporation  into  the  plan,  at  least  for  further  discussion. 
Keeping  track  of  these  options  is  a  very  formidable  task  for  the  planners.  The 
options  often  suffer  the  fate  of  being  lost  due  to  the  distractions  and  the 
non-linear  nature  of  the  planning  process.  If  a  viable  option  is  generated, 
only  to  be  forgotten  due  to  a  change  of  focus  of  the  planning  topic,  it  may 
never  be  retrieved  later.  If  it  is  retrieved,  it  may  be  only  through  a  com¬ 
plete  reconstruction  of  the  earlier  discussions  at  the  expense  of  time  and 
efficiency. 

Could  a  DSS  be  used  to  help  track  these  generated  options?  It  was  observed 
that  the  planners  generally  focus  the  planning  task  along  two  lines.  First, 
they  concentrate  upon  specific  topics  of  discussion  (obstacles,  avenues  of 
advance  and  approach,  positions,  etc.).  Second,  they  usually  limited  their 
focus  to  portions  of  the  battle  sector  that  generally  were  defined  by  the  ter¬ 
rain  and  existing  obstacles  (rivers,  cities,  etc.).  The  result  is  that  the 
plan  tended  to  develop  by  topic  within  specific  geographical  sub-sectors  of  the 
battle  area.  For  example,  the  planners  would  develop  a  plan  for  the  placement 
of  obstacles  along  the  high  speed  avenue  of  approach  within  the  area  marked  "A" 
on  the  map  in  Figure  6.  They  did  not  plan  for  the  placement  of  obstacles 
within  the  entire  battle  sector  (the  entire  area  of  the  map)  all  at  once. 

This  provides  us  with  a  general  model  of  how  the  planners  were  working  with 
the  map  and  their  focus  with  respect  to  the  tasks  at  hand.  Building  from  this 
model  we  can  make  some  recommendations  for  the  development  of  the  DSS.  The 
following  is  a  description  of  how  one  6uch  DSS  recommendation  could  work. 

The  planners  work  with  a  digitized  representation  of  the  map  of 
their  battle  sector.  During  the  initial  planning,  they  outline 
key  geographical  areas  of  their  sector  with  a  lightpen  (such  as 
section  "A,"  "B,"  "C,"  "D,"  etc.,  of  Fig.  6).  These  are  stored  as 
"scratch  pad"  areas  for  later  reference  by  the  computer.  When  the 
planners  begin  a  discussion  about  a  particular  topic  within  a 
specific  area  of  the  battle  sector  (e.g. ,  placement  of  obstacles 
within  section  "A"  of  Fig.  6)  they  use  the  lightpen  to  indicate  they 
want  to  work  with  a  scratch  pad  and  touch  this  section  of  the  map. 
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The  planners  can  choose  to  leave  this  selected  scratchpad  area  the 
same  size,  or  enlarge-  it  while  they  work  on  it.  As  the  planners 
formulate  an  option  for  the  placement  of  obstacles,  they  pull  icons 
from  the  top  of  the  screen  and  place  them  where  they  want  them  on  the 
scratchpad.  When  they  are  finished  they  simply  exit  and  the  computer 
automatically  saves  this  scratchpad  under  a  name  predetermined  by  the 
nature  of  the  icons  used  (obstacles,  routes,  etc.).  Later,  when  the 
planners  return  to  this  particular  scratchpad,  they  will  initially  be 
shown  a  list  of  the  automatically  saved  pads,  sequentially  named  by 
the  icon  identifiers  (e.g. ,  obstaclel,  obstacle?,  minesl,  FASCAM1, 
routesl,  positionl).  This  reminds  the  planners  of  options  they 
formulated  earlier  within  this  specific  geographical  area.  They  can 
retrieve  any  of  these  previously  formulated  options  or  go  back  into 
the  scratchpad  to  develop  new  alternatives.  Anytime  the  planners 
desire,  an  option  can  be  transferred  from  scratch  pad  status  to  the 
"real"  map  for  inclusion  in  the  plan. 

Although  this  is  a  sophisticated  mechanism,  most  of  the  supporting  technol¬ 
ogy  for  it  already  exists  on  PC-style  machines. 


Figure  5.  Option  formulation  map. 
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Memory  aids  for  progressively  deepened  parts  of  the  plan.  The  maps  of  pro¬ 
gressive  deepening  chart  the  paths  followed  by  the  planners  while  they  worked 
on  particular  pieces  of  the  plan.  These  charts  indicated  that  the  planners 
needed  to  reconstruct  "where  they  left  off"  when  they  returned  later  to  the 
particular  topic.  This  was  not  the  most  efficient  way  to  "grow  a  plan,"  espe¬ 
cially  if  the  planners  were  working  under  time  pressure.  A  DSS  using  the 
scratch  pad  mechanism  discussed  above  could  provide  assistance  to  the  planners 
by  allowing  retrieval  of  previously  planned  (deepened)  segments  so  they  can 
more  easily  reconstruct  "where  they  left  off."  This  could  result  in  a  more 
efficient  use  of  their  limited  time  and  reduce  the  amount  of  lost  information. 

Implications  for  Training 

Management  activities.  Battle  management  planning  often  appears  confused, 
overloaded  and  chaotic.  Anyone  who  examines  the  transcripts,  progressive  deep¬ 
ening  sheets,  and  other  results  will  be  struck  by  how  easily  distracted  the 
planners  actually  were.  Although  there  is  no  comparative  evidence  that  a  more 
controlled  procedure  would  produce  better  results,  it  seems  rational  to  predict 
that  reductions  in  repetitive  behavior  and  concept  loss  would  help. 

This  emphasizes  the  need  for  the  planners  to  fully  understand  exactly  how. 
the  process  unfolds,  including  all  the  distractions,  pitfalls,  and  diversions. 
This  is  especially  true  if  we  want  them  to  be  able  to  manage  this  process  as 
well  as  possible.  Many  of  the  distractions  were  subtle  and  they  occurred  with¬ 
out  any  verbalized  awareness  on  the  part  of  the  planners,  further  handicapping 
the  person  responsible  for  managing  the  planning  session. 

It  is  hypothetical  whether  and  what  kinds  of  training  would  improve  process 
management,  but  we  offer  the  following  as  possibilities.  Training  for  these 
individuals  needs  to  include  sessions  which  increase  their  awareness  of  the 
process  itself,  not  just  their  understanding  of  the  military  issues  that  they 
are  to  incorporate  in  the  final  product.  It  may  be  that  just  a  couple  of  hours 
examining  selected  parts  of  transcripts,  progressive  deepening  charts  and  data 
concerning  the  nature  of  transitions  between  segments  of  planning  may  be  enough 
to  increase  their  awareness  of  the  potential  problems.  This  in  itself  may  be 
enough  to  improve  performance.  On  the  other  hand,  It  may  be  necessary  to  also 
include  specific  training  on  how  to  actively  maintain  control  of  the  process. 

Modification  of  CDM  for  after-action  reviews.  It  has  been  amply  demonstra¬ 
ted  that  immediate  feedback  is  beneficial  to  learning.  It  is  possible  that 
some  of  our  analytical  methods  could  be  modified  into  an  abbreviated  format. 

ThlB  would  be  such  that  a  domain-knowledgeable,  trained  abserver(s)  could  chart 
the  events  as  they  occur  along  dimensions  that  would  permit  review  either  imme¬ 
diately  following  or  at  planned  breaks  during  an  exercise  in  the  form  of  an 
After-Action  Review  (AAR).  In  this  fashion,  the  planners  could  receive  rela¬ 
tively  rapid  feedback  about  their  actual  planning  style  and  how  they  direct  the 
decision  making  process.  Realistically,  the  topics  where  feedback  is  desired 
would  have  to  be  predetermined  and  would  need  to  be  limited  to  two  or  three 
issues,  for  example,  information  acquisition,  information  use,  option  formula¬ 
tion,  the  nature  of  transitions  between  topics  of  discussion,  reconstruction  of 
previously  covered  options,  use  of  available  resources  and/or  progressive  deep¬ 
ening. 
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In  this  setting,  the  observer(s)  would  forego  attempting  to  thoroughly  map 
the  entire  process  and  would  instead  concentrate  on  tracking  the  specifically 
pre-identif ied  issues  of  interest.  Ideally,  the  analyzed  data  would  be  ready 
within  30  minutes  for  presentation  at  the  AAR.  Thi6  could  include  playback,  of 
specific  audio  (or  video)  taped  segments  to  demonstrate  particular  points. 

What  we  are  proposing  is  a  simplified  variation  of  these  methods  for  di¬ 
rectly  tracking  key  aspects  of  the  progressive  deepening  in  a  natural  setting. 
The  products  could  then  be  used  for  immediate  feedback  for  training. 

We  would  expect  the  results  to  be  helpful  in  the  following  training  areas: 

•  The  participants  could  see  the  process  of  how  they  managed  the 

discussions  and  the  group  by  how  linearly  both  progressed.  That  is, 
while  going  to  A,  sidetracks  X,  Y,  and  Z  were  also  taken. 

•  It  would  provide  a  framework  to  critique  how  they  handled  barriers 
encountered.  That  is,  what  were  the  barriers  they  did  not  consider,  what 
were  the  barriers  they  became  hung-up  on,  where  information  was  needed 
but  not  gone  after,  ways  they  could  have  overcome  barriers  they  were  or 
were  not  aware  of,  etc. 

We  intend  to  develop  such  an  after-action  review  technique  as  a  follow-on 

project.  The  description  of  this  technique  and  the  results  of  its  evaluation 

will  be  covered  in  a  separate  report. 

CDM  data  training  modules.  In  its  non-abbreviated  form,  the  CDM  can  be  used 
to  collect  materials  which  could  be  compiled  into  training  modules  for  class¬ 
room  settings.  The  modules  could  include  selected  non-attributable  excerpts 
from  transcripts.  These  transcripts  would  contain  examples  of  different  types 
of  transitions  between  topics,  examples  of  options  and  information  being  "lost" 
due  to  distractions,  progressive  deepening  charts  to  graphically  demonstrate 
the  nature  of  "growing  a  plan."  They  may  also  contain  examples  of  distractions, 
redirections,  and  decisions,  examples  of  places  where  additional  Information  is 
needed  and  whether  or  not  the  planners  actively  sought  out  this  information, 
and  so  on.  The  modules  could  be  tailored  for  specific  groups  or  constructed  in 
a  more  generic  fashion  for  broader  audiences. 

Develop  a  performance  metric  for  group  decision  making.  In  general,  the  CDM 
is  employed  ss  a  means  of  determining  the  nature  of  the  decision  processes  and 
not  as  an  evaluative  tool  with  regards  to  the  quality  of  the  decisions.  It 
could,  however,  a®  used  to  compile  a  metric  of  performance  measures  to  evaluate 
the  performance  of  planning  groups. 

These  measures  could  be  based  upon  Information  available  from  the  applica¬ 
tion  of  the  CDM.  Examples  include: 

•  How  often  they  sought  information  when  they  needed  it.  How  often  they 
failed  to  seek  it. 
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•  Deliberate  versus  accidental  inclusion  of  options*  Many  options  are  gen** 
erated  which  become  part  of  the  plan.  A  select  number  of  these  are  in¬ 
corporated  through  the  conscious  desire  of  the  planners.  It  appears  that 
others  become  part  of  the  plan  almost  by  default  (i.e.,  the  planners 
become  distracted  and  never  return  to  their  consideration  of  the  option, 
never  make  any  type  of  decision  regarding  it,  yet  it  still  finds  its  way 
into  the  plan).  The  ratio  of  deliberate  versus  accidental  options  in¬ 
cluded  in  the  final  plan  could  be  determined. 

•  The  number  of  "loose  threads"  that  are  still  dangling  when  the  plan  is 
finalized  (e.g. ,  no  decision  was  ever  made  about  how  to  obstruct  this 
avenue  of  approach). 

•  How  the  planners  prioritized  the  use  of  the  time  available  with  respect  to 
'tasks  to  be  accomplished.  The  battle  managers  we  observed  simply  jumped 
into  the  process,  started  generating  and  evaluating  options,  and  finished 
when  they  ran  out  of  time.  He  did  not  see  any  attempts  to  manage  the  use 
of  time.  We  did  not  observe  anyone  saying  "we’ve  got  5  hours  here,  and 
issues  X,  Y,  and  Z  are  the  most  important,  so  let's  start  with  X  but  try 
to  reach  closure  within  40  minutes."  While  it  might  be  impossible  to  train 
people  to  be  that  rational,  and  it  may  even  prove  counterproductive, 
clearly  there  can  be  a  better  sense  of  directing  the  discussion  to  fit 

the  priorities.  If  the  ability  to  manage  time  comes  from  experience, 
perhaps  it  can  be  trained  more  effectively. 

Summary 

We  studied  three  different  battle  management  exercises  in  order  to  determine 
if  the  CDM  interviews  could  be  applied  within  a  C2  setting,  and  to  try  new 
techniques  for  collecting  and  analyzing  data.  We  found  that  the  CDM  interviews 
were  compatible  with  the  constraints  of  military  exercises.  We  also  learned 
that  certain  CDM  interview  questions  were  redundant  with  information  that  could 
be  collected  by  direct  observation  and  taping  of  the  planning  sessions.  We 
also  developed  micro-analytical  techniques  of  data  reduction:  a  five-hour 
planning  session  was  analyzed  into  64  segments  and  these  were  further  catego¬ 
rized  In  terms  of  goals  and  processes.  More  interesting  were  the  transitions 
between  segments.  Fully  one-third  were  due  to  out-of-contexi  interruptions, 
and  planning  rarely  resumed  from  the  point  of  the  Interruption. 

We  analyzed  information  sources  and  found  that  93%  of  the  information  con¬ 
sidered  was  directly  available  to  the  battle  managers.  There  was  little  active 
pursuit  of  data,  and  not  a  single  request  for  material  from  S2  (Intelligence) 
following  the  S2  intelligence  update,  which  occurred  two  to  three  hours  prior 
to  the  main  planning.  This  is  even  more  interesting  in  view  of  the  fact  that 
the  planners  repeatedly  complained  about  the  lack  of  knowledge  of  the  eneoy 
plans,  intentions  and  attack  routes. 

The  data  we  collected  revealed  a  number  of  features  of  group  decision  making 
by  these  battle  managers.  We  confirmed  earlier  findings  about  the  reliance  on 
recognition  decisions  and  the  infrequency  of  analytical  decisions.  Even  though 
doctrine  calls  for  concurrent  deliberation,  we  found  little  evidence  for  it. 


Instead,  we  saw  a  "satisficing"  strategy  marked  by  a  one-at-a-time  search  for 
an  option  that  would  work  and  had  no  pitfalls  associated  with  it.  Therefore, 
training  that  emphasizes  generating  and  evaluating  sets  of  options  may  be 
counterproductive.  Such  training  does  not  describe  the  performance  of  the 
decision  makers  we  have  observed,  and  it  may  reduce  the  credibility  of  the 
training  program.  In  addition,  decision  support  systems  that  promote  the  use 
of  formal  analytical  methods  are  inconsistent  with  the  decision  strategies  we 
observed. 

We  have  proposed  alternative  concepts  for  decision  support  systems  and 
training  that  we  feel  wil?  be  more  consistent  with  the  decision  strategies  that 
are  being  used. 
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APPENDIX  A 

INTERVIEW  GUIDES  FOR  OPERATIONS  AND  INTELLIGENCE 


G3/..3  PROBES:  Based  on  G3/S3  plans  to  this  point* 

1.  What  are  the  goals  for  the  plan?  Were  there  alternative  goals? 

2.  What  are  the  most  important  aspects/concerns  you  have  about  the  current 
state  of  the  plan?  (be  alert  for  X  vs  Y). 

'2a*  How  would  you  rank  these  concerns  in  order  of  importance  for  accomplishing 
the  goals  of  the  plan? 

3.  Did  you  consider  other  options/alternatives  for  the  plan  to  this  point? 

Was  experience  level  a  factor  in  the  choice  of  alternatives?  How? 

4.  What  key  factors  do  you  feel  helped  or  hurt  your  planning  process? 

5*  Which  factor s/assumptions ,  if  different ,  are  critical  enough  to  make  the 
plan  inoperative  or  Inexecutable?  (Xs  and  Ys) 

5a.  Rank  these  factors  in  order  of  importance  for  accomplishing  the  goals  of 
the  plan. 

6a.  Was  there  any  material  from  briefings  which  you  felt  was  particularly 
helpful  to  you  in  your  job  as  G3/S3?  Elaborate  on  reasons  why  helpful. 

6b.  Did  any  of  this  material  from  the  briefings  cause  you  to  notice  anything 
which  you  hadn't  previously  considered? 


G2/S2  PROBES:  Based  on  G2/S2  estimate  of  enemy  plan  to  this  point* 

1.  What  are  the  goals  for  the  plan?  Were  there  alternative  goals? 

2.  What  are  the  most  important  aspects/ concerns  you  have  about  the  current 
state  of  the  plan?  (be  alert  for  X  vs  Y). 

2a.  How  would  you  rank  these  concerns  in  order  of  importance  for  accomplishing 
the  goals  of  the  plan? 

3.  Did  you  consider  other  options/alternativee  for  the  plan  to  this  point? 

Was  experience  level  a  factor  in  the  choice  of  alternatives?  How? 

4.  What  are  the  key  pieces  of  information  you,  as  G2/S2,  possess?  (Xs  and 
Ys). 

4a,  How  would  you  rank  these  pieces  of  information  in  order  of  importance  to 

the  plan  (our  plan  or  the  enemy's  plan???)? 

5.  What  key  pieces  of  information  do  you  feel  are  missing?  (Xs  and  Ys). 
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5a*  How  would  you  rank  these  missing  pieces  of  information  in  order  of 
importance  to  the  (ours  or  theirs??)  plan? 

6a.  How  much  of  all  this  information  do  you  plan  to  report  to  G3/S3? 

7.  What  key  factors  do  you  feel  either  helped  or  hurt  your  planning  process 

8.  Which  factors/assumptions,  if  different,  are  critical  enough  to  make  the 
plan  inoperative  or  inexecutable?  (Xs  and  Ys) 

9a*  Was  there  any  material  from  briefings  which  you  felt  was  particularly 
helpful  to  you  in  your  job  as  G2/S2?  Elaborate  on  reasons  why  helpful. 

9b.  Did  any  of  this  material  from  the  briefings  cause  you  to  notice  anything 
which  you  hadn't  previously  considered? 

Note:  All  probes  with  sub-letters  (9a,  9b,  5a,  etc.)  are  to  be  omitted  first 
if  available  interview  time  is  short. 
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APPENDIX  B 


SAMPLE  OF  TRANSCRIPT  WITH  SEGMENTS  AND 
REFERENCE  UNITS  INDICATED 


Segment  Divider: 

Reference  Unit  Divider:  - — 

Reference  Units  Numbered  Along  Left  Margin. 
Capital  Letter/Colon  (Z:)  Indicates  Speaker. 


165  Z:  Options  sheet,  don't  blow  all  the  bridges,  we  don't  need  to  blow 
all  the  bridges,  the  ones  to  the  north  the  ones  to  the  south,  why 
not  try  to  force  them  to  come  through  the  center  and  the 
engagement  area's  here. 


166  Y:  1  don't  want  to  do  that  because  I  don't  have  anyone  to  stop  them 

when  they  come  through  there,  that's  to  give  them  an  easy  access, 
1  want  to  make  it  as  hard  as  possible,  I  just  don't  want  to 
give. .. 

167  Z:  Their  doctrine  is  they  wouldn't  take  off  along  here,  that's  some 

fairly  steep  terrain  through  there,  they  take  the  high  Bpeed. 


168  Y:  But  they  are  also,  once  our  recon  pulls  back,  they're  going  to 

have  their  scouts  forward  looking  for  those  crossing  and  they're 
going  to  find  them  quickly  and  then  they're  going  to  see  these 
blown  and  these  blown,  they're  going  to  have  to  move  through  here 
and  they're  going  to  go  that  road  as  quickly  as  possible,  slowing 
them  down  is  what  I  want  to  do,  that’s  why  I'm  blowing  those  and 
I'm  going  to  leave  these  open  for  our  guys  to  come  through. 


169  Z:  That's  where  right  in  there  and  have  them  searching  for  a  way 

around  would  be  the  place  to  drop  your  FASCAM,  force  them  down  here. 

Y:  Ok,  but  then  we're  going  to  go  back. 

2:  Option. 

170  Y:  I  agree,  that's  an  excellent  spot,  but  it's  not  going  to  do  us  any 

good  on  our  pass,  cause  there's  not  going  to  be  anyone  there  to 
see  it,  unless  we  got  a  scout  in  town,  we  could  tell  them  there  is 
one  scout  we  dropped  off  in  the  town  or  something  like  that. 

Z:  What  I'm  saying  is,  this  is  preferred,  we  just  don't  in  that  area, 

know  what  I  mean,  we  drop  it  in  there,  400  x  400  field. 


171  U:  You  were  talking  that  the  bridges  should  already  be  precratered, 
it's  just  a  matter  of  bringing  up  the  stuff  that's  just  means  we 
need  trucks.  Does  the  engineer  know  that? 
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Y:  Yea,  we  were  talking  about  that. 

Z:  400  x  400  field. 

Y:  We  were  talking  about  it,  so  .  .  • 

Zs  400  x  400  field  here,  right,  that's  keep  them  from  going  that  way 

for  damn  sure. 

W:  Sure  will 


Y:  And  then  we  need  to  deny  them  fast  coming  across  this  road. 

Z:  Then  he  can  Jerk  his  effort  there,  go  a  little  bit  down  here. 

Y:  The  only  effort  there  is  to  drop  the  charges  already  have 

precratered  bridge,  that's  all  it  is,  is  just  to  knock  the  bridge 
out,  however  we  do  it,  whether  we  use.  . 

Z:  1  thought  that  was.  •  . 

Y:  That's  a  bridge,  I  pull  it  on  mines,  little  boxes. 

Zs  Well,  I  look  at  minefields  there  I  say  well,  hell,  you  move  that 
minefield  right  down  here  and  he  will  go  ahead  with  the  high  6peed 
avenue,  once  he's  forced  to  the  high  speed  avenue  he'll  take  it.  . 

Y:  Just  ignore  the  boxes  then,  what  that  is  a  bridge  and  that's  a 

crater,  that's  all  1  put  there,  cause  1  got  to  brief  it  so  1  just 
put  those  there  to  remind  me. 

Z;  Col.  will  hit  you  on  it. 

Y:  I'll  probably  get  rid  of  them,  cause  he  already  told  me  about 

that  • 

Z:  Col.  likes  detail,  I've  been  working  with  him  for  years. 


Alright,  let's  go  ahead  and  put  a  fast  unit  in  there  400  x  400  right 
there,  you  want  to  leave  these  open? 

Y:  No,  I  don't  want  to  leave  them  open* 

Z:  Then  it  doesn't  do  them  any  good  to  throw  FASCAMs  there. 

Y:  Sure  it  does,  cause  they  may  come  across  and  then  they  hit  another 

mine. 
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176  Z:  What  I’m  saying  is,  if  you  blow  the  bridges,  alright,  he's  going 
to  say  screw  that,  he's  going  to  look  for  a  bridge,  wouldn't  do  me 
any  good  to  throw  it  there. 

Y :  How  many  do  we  have 

Z:  Cause  it  doesn't  tie  in  with  anything. 

Y:  How  many  FASCAMs  do  you  have  available. 

W:  Two.  Possible  3.  So  to  answer  your  question  we  need  to  find  those 

2  FASCAM  spots. 


177  Z:  Oh  he  will  want  to  use  them 

Y:  I  know. 

Z;  Doesn't  mean  that  he's  going  to  come  in  and  just  blow  this  plan 
all  apart. 

Y:  He  blows  it  apart  that's  fine. 

Zi  FASCAM  in  a  big  open  area  like  that  what  does  it  do,  does  it  tie 
in  any  engineer  effort,  does  it  block  passage  to  the  plans,  does 
it  channelize  or  deny  him  area,  no.  Take  care  of  all  of  them  your 
just  mechanic. 

Y:  That's  ok,  I'm  learning,  I  got  more  advice  than  1  know  what  to  do 

with,  other  than  that. 


178  W:  Just  tell  me  where  you  want  my  company. 

Y:  Originally  we're  going  to  set  you  up  right  in  here,  the  3b  will  be 

Charlie  is  the  way  it  goes,  alphas  will  be  Is,  bravos  will  be  2s 
and  Charlie's  will  be  3s.  See  initially  be  here,  the  engagement 
area  is  here,  Col,  want  to  defend  in  depth,  then  I've  worked  out  a 
route,  obviously  when  you  get  out  there  you  will  look  for  the  best 
route  back  to  attack. 


179  U:  My  whole  company  out  there? 

Y:  He  wants  a  company  positions  here,  if  you  want  to  maybe  put  two 

forward  and  then  leave  one  back  here  we'll  talk  about  it. 

U:  Where's  that  counterrecon  going? 

180  Y:  You're  going  to  come  back  here,  basically  I  don't  want  to  put 

anything  in  here  other  than  just  you  might  have  a  platoon  up  here, 
we  were  talking  about  that  with  two  captains.  Put  a  section  or 
platoon  up  here  if  they  use  this  road  take  them  out  in  the  open 
area,  maybe  put  a  crater  in  here.  I  put  you  over  here  because  if 
they  move  through  the  woodline,  as  they  come  out  of  the  woodline 
you  may  be  able  to  get  some  shots  with  your  defensive  position,  I 
need  you  to  go  out  and  look  at  that  and  see  if  that's  going  to  be 
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viable,  with  your  final  position  right  here  using  the  RR,  I'm 
going  to  start  ditching  effort  starting  from  the  north,  working 
down,  so  this  will  be  your  main  engagement  area  here,  bravo  be  in 
here,  we'll  have  alpha  and  delta,  and  these  are  fall  back,  there's 
is  no  real  fightable  terrain  in  here,  this  i6  what  we  have  to  deny 
them,  right  here,  autobahn.  This  is  going  to  be  all  bogged  up. 

181  W:  So  why  doesn't  someone  take  this  from  me  in  the  middle  so  1  can 

take  the  north  sector  and  worry  about  the  north.  Cause  you  got 
these  guys  doing  nothing  up  front  at  least  from  the  graphics  it 
looks  that  way.  Let  them  worry  about  picking  up  the  guys  in  the 
middle  and  they  can  go  back  to  positions  you  got  me  way.  .  . 


182  Y;  The  reason  I  got  them  set  here,  see  this  plan  obviously  once  it 
starts  it  may  change,  the  reason  I  have  bravo  here  is  they're  the 
anchor  for  the  platoon,  see  their  center  sector. 
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APPENDIX  C:  CONTENT  ANALYSIS  DATA  MAP  EXCERPT 
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